[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439250192320-29290-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:07:30 dist-test-slave-xt4k krb5kdc[29302](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:07:30 dist-test-slave-xt4k krb5kdc[29302](info): set up 2 sockets
Aug 11 09:07:30 dist-test-slave-xt4k krb5kdc[29302](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260811 09:07:31.623018 29290 mini_kdc.cc:121] Time spent starting KDC: real 1.262s user 0.002s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439250192320-29290-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439250192320-29290-0/krb5kdc/test-user.keytab.
Aug 11 09:07:31 dist-test-slave-xt4k krb5kdc[29302](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439251, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:07:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:07:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.154.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.154.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.154.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab.
Entry for principal kudu/127.28.154.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab.
I20260811 09:07:31.830111 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439250192320-29290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439250192320-29290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439250192320-29290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.154.190:34837
--webserver_interface=127.28.154.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab
--principal=kudu/127.28.154.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:38387
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.154.190:34837
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:07:32.269730 29324 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:07:32.270159 29324 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:07:32.270308 29324 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:07:32.278239 29324 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:07:32.278367 29324 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:07:32.278426 29324 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:07:32.278508 29324 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:07:32.278580 29324 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:07:32.290792 29324 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:38387
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439250192320-29290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439250192320-29290-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.154.190:34837
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab
--principal=kudu/127.28.154.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.154.190:34837
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.154.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439250192320-29290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:07:32.293148 29324 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:07:32.295161 29324 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:07:32.316896 29337 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:07:32.316923 29340 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:07:32.319286 29332 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:07:32.320523 29324 server_base.cc:1034] running on GCE node
I20260811 09:07:32.321748 29324 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:07:32.324384 29324 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:07:32.327798 29324 hybrid_clock.cc:648] HybridClock initialized: now 1786439252327720 us; error 158 us; skew 500 ppm
I20260811 09:07:32.328578 29324 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:07:32 dist-test-slave-xt4k krb5kdc[29302](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439252, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:07:32.332998 29324 init.cc:377] Logged in from keytab as kudu/127.28.154.190@KRBTEST.COM (short username kudu)
I20260811 09:07:32.335887 29324 webserver.cc:460] Webserver started at http://127.28.154.190:34429/ using document root <none> and password file <none>
I20260811 09:07:32.337196 29324 fs_manager.cc:359] Metadata directory not provided
I20260811 09:07:32.337342 29324 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:07:32.337857 29324 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:07:32.342317 29324 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439250192320-29290-0/minicluster-data/master-0/data/instance:
uuid: "8b9e43cb92ec434881a925f25180ec58"
format_stamp: "Formatted at 2026-08-11 09:07:32 on dist-test-slave-xt4k"
server_key: "def4b1298ab9b1584ec41b796a649d95"
server_key_iv: "1bcdc30f63d79fd797cf4caef291dbc5"
server_key_version: "encryptionkey@0"
I20260811 09:07:32.343161 29324 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439250192320-29290-0/minicluster-data/master-0/wal/instance:
uuid: "8b9e43cb92ec434881a925f25180ec58"
format_stamp: "Formatted at 2026-08-11 09:07:32 on dist-test-slave-xt4k"
server_key: "def4b1298ab9b1584ec41b796a649d95"
server_key_iv: "1bcdc30f63d79fd797cf4caef291dbc5"
server_key_version: "encryptionkey@0"
I20260811 09:07:32.352665 29324 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.011s sys 0.000s
I20260811 09:07:32.359436 29349 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:07:32.361773 29324 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.006s sys 0.000s
I20260811 09:07:32.362129 29324 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439250192320-29290-0/minicluster-data/master-0/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439250192320-29290-0/minicluster-data/master-0/wal
uuid: "8b9e43cb92ec434881a925f25180ec58"
format_stamp: "Formatted at 2026-08-11 09:07:32 on dist-test-slave-xt4k"
server_key: "def4b1298ab9b1584ec41b796a649d95"
server_key_iv: "1bcdc30f63d79fd797cf4caef291dbc5"
server_key_version: "encryptionkey@0"
I20260811 09:07:32.362466 29324 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439250192320-29290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439250192320-29290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439250192320-29290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:07:32.391932 29324 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:07:32.397927 29324 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:07:32.398331 29324 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:07:32.427273 29324 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.190:34837
I20260811 09:07:32.427294 29442 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.190:34837 every 8 connection(s)
I20260811 09:07:32.429957 29324 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
I20260811 09:07:32.434171 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 29324
I20260811 09:07:32.434604 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439250192320-29290-0/minicluster-data/master-0/wal/instance
I20260811 09:07:32.437855 29290 external_mini_cluster.cc:1428] Setting key f4de9b03a0939b7264ee3153404eb7bf
I20260811 09:07:32.438815 29443 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:07:32.457945 29443 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b9e43cb92ec434881a925f25180ec58: Bootstrap starting.
I20260811 09:07:32.465893 29443 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8b9e43cb92ec434881a925f25180ec58: Neither blocks nor log segments found. Creating new log.
I20260811 09:07:32.467821 29443 log.cc:824] T 00000000000000000000000000000000 P 8b9e43cb92ec434881a925f25180ec58: Log is configured to *not* fsync() on all Append() calls
I20260811 09:07:32.472381 29443 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b9e43cb92ec434881a925f25180ec58: No bootstrap required, opened a new log
Aug 11 09:07:32 dist-test-slave-xt4k krb5kdc[29302](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439251, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.154.190@KRBTEST.COM
I20260811 09:07:32.483479 29443 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8b9e43cb92ec434881a925f25180ec58 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b9e43cb92ec434881a925f25180ec58" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 34837 } }
I20260811 09:07:32.484031 29443 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8b9e43cb92ec434881a925f25180ec58 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:07:32.484212 29443 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8b9e43cb92ec434881a925f25180ec58 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b9e43cb92ec434881a925f25180ec58, State: Initialized, Role: FOLLOWER
I20260811 09:07:32.485579 29443 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8b9e43cb92ec434881a925f25180ec58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b9e43cb92ec434881a925f25180ec58" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 34837 } }
I20260811 09:07:32.486027 29443 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8b9e43cb92ec434881a925f25180ec58 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:07:32.486322 29443 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8b9e43cb92ec434881a925f25180ec58 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:07:32.486582 29443 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8b9e43cb92ec434881a925f25180ec58 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:07:32.489086 29447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:32.445647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50908 (local address 127.28.154.190:34837)
0811 09:07:32.446920 (+ 1273us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:32.446939 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:32.450423 (+ 3484us) server_negotiation.cc:407] Connection header received
0811 09:07:32.462738 (+ 12315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:32.462770 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:32.463718 (+ 948us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:32.464291 (+ 573us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:32.467877 (+ 3586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:32.469447 (+ 1570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:32.471284 (+ 1837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:32.472025 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:32.472297 (+ 272us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:32.478542 (+ 6245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:07:32.478617 (+ 75us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:07:32.478660 (+ 43us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:07:32.478716 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:07:32.482606 (+ 3890us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:32.483581 (+ 975us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:32.483588 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:32.483594 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:32.483833 (+ 239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:32.484407 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:32.484420 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:32.484422 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:32.485178 (+ 756us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:07:32.485652 (+ 474us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:32.486729 (+ 1077us) server_negotiation.cc:299] Negotiation successful
0811 09:07:32.487140 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":852,"thread_start_us":397,"threads_started":1}
I20260811 09:07:32.490166 29443 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8b9e43cb92ec434881a925f25180ec58 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b9e43cb92ec434881a925f25180ec58" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 34837 } }
I20260811 09:07:32.491448 29443 leader_election.cc:302] T 00000000000000000000000000000000 P 8b9e43cb92ec434881a925f25180ec58 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8b9e43cb92ec434881a925f25180ec58; no voters:
I20260811 09:07:32.492432 29443 leader_election.cc:288] T 00000000000000000000000000000000 P 8b9e43cb92ec434881a925f25180ec58 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:07:32.492842 29451 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8b9e43cb92ec434881a925f25180ec58 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:07:32.495434 29451 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8b9e43cb92ec434881a925f25180ec58 [term 1 LEADER]: Becoming Leader. State: Replica: 8b9e43cb92ec434881a925f25180ec58, State: Running, Role: LEADER
I20260811 09:07:32.495985 29443 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8b9e43cb92ec434881a925f25180ec58 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:07:32.496313 29451 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8b9e43cb92ec434881a925f25180ec58 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b9e43cb92ec434881a925f25180ec58" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 34837 } }
I20260811 09:07:32.501950 29456 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b9e43cb92ec434881a925f25180ec58 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8b9e43cb92ec434881a925f25180ec58. Latest consensus state: current_term: 1 leader_uuid: "8b9e43cb92ec434881a925f25180ec58" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b9e43cb92ec434881a925f25180ec58" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 34837 } } }
I20260811 09:07:32.502508 29456 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b9e43cb92ec434881a925f25180ec58 [sys.catalog]: This master's current role is: LEADER
I20260811 09:07:32.502992 29452 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b9e43cb92ec434881a925f25180ec58 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8b9e43cb92ec434881a925f25180ec58" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b9e43cb92ec434881a925f25180ec58" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 34837 } } }
I20260811 09:07:32.503345 29452 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b9e43cb92ec434881a925f25180ec58 [sys.catalog]: This master's current role is: LEADER
I20260811 09:07:32.507510 29476 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:07:32.516405 29476 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:07:32.534752 29476 catalog_manager.cc:1347] Generated new cluster ID: 01b9096e29b04cf5a78b5857cda5d4ba
I20260811 09:07:32.534914 29476 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:07:32.547600 29476 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:07:32.549860 29476 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:07:32.575320 29476 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8b9e43cb92ec434881a925f25180ec58: Generated new TSK 0
I20260811 09:07:32.576568 29476 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.28.154.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.154.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.154.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.129.keytab.
Entry for principal kudu/127.28.154.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.129.keytab.
I20260811 09:07:32.639572 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439250192320-29290-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439250192320-29290-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439250192320-29290-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439250192320-29290-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.154.129:0
--local_ip_for_outbound_sockets=127.28.154.129
--webserver_interface=127.28.154.129
--webserver_port=0
--tserver_master_addrs=127.28.154.190:34837
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.129.keytab
--principal=kudu/127.28.154.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:38387
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:07:33.042519 29496 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:07:33.042841 29496 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:07:33.042941 29496 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:07:33.051038 29496 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:07:33.051167 29496 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:07:33.051285 29496 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.154.129
I20260811 09:07:33.062449 29496 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:38387
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439250192320-29290-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439250192320-29290-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.129.keytab
--principal=kudu/127.28.154.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.154.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439250192320-29290-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.154.129
--webserver_port=0
--tserver_master_addrs=127.28.154.190:34837
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.154.129
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439250192320-29290-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:07:33.063813 29496 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:07:33.065557 29496 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:07:33.080940 29504 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:07:33.080984 29508 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:07:33.081003 29505 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:07:33.082252 29496 server_base.cc:1034] running on GCE node
I20260811 09:07:33.082968 29496 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:07:33.084048 29496 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:07:33.085291 29496 hybrid_clock.cc:648] HybridClock initialized: now 1786439253085234 us; error 45 us; skew 500 ppm
I20260811 09:07:33.085757 29496 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:07:33 dist-test-slave-xt4k krb5kdc[29302](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439253, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:07:33.089895 29496 init.cc:377] Logged in from keytab as kudu/127.28.154.129@KRBTEST.COM (short username kudu)
I20260811 09:07:33.091662 29496 webserver.cc:460] Webserver started at http://127.28.154.129:44433/ using document root <none> and password file <none>
I20260811 09:07:33.092466 29496 fs_manager.cc:359] Metadata directory not provided
I20260811 09:07:33.092607 29496 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:07:33.092953 29496 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:07:33.096199 29496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439250192320-29290-0/minicluster-data/ts-0/data/instance:
uuid: "6239333fb48242fa8a85c7aa40de0745"
format_stamp: "Formatted at 2026-08-11 09:07:33 on dist-test-slave-xt4k"
server_key: "057ea7dde29ec9075b8415b62ac26c57"
server_key_iv: "e18e6e6e8a991c0adfde025174652aa0"
server_key_version: "encryptionkey@0"
I20260811 09:07:33.097250 29496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439250192320-29290-0/minicluster-data/ts-0/wal/instance:
uuid: "6239333fb48242fa8a85c7aa40de0745"
format_stamp: "Formatted at 2026-08-11 09:07:33 on dist-test-slave-xt4k"
server_key: "057ea7dde29ec9075b8415b62ac26c57"
server_key_iv: "e18e6e6e8a991c0adfde025174652aa0"
server_key_version: "encryptionkey@0"
I20260811 09:07:33.103907 29496 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260811 09:07:33.108315 29516 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:07:33.109973 29496 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 09:07:33.110239 29496 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439250192320-29290-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439250192320-29290-0/minicluster-data/ts-0/wal
uuid: "6239333fb48242fa8a85c7aa40de0745"
format_stamp: "Formatted at 2026-08-11 09:07:33 on dist-test-slave-xt4k"
server_key: "057ea7dde29ec9075b8415b62ac26c57"
server_key_iv: "e18e6e6e8a991c0adfde025174652aa0"
server_key_version: "encryptionkey@0"
I20260811 09:07:33.110535 29496 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439250192320-29290-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439250192320-29290-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439250192320-29290-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:07:33.145391 29496 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:07:33.150702 29496 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:07:33.151175 29496 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:07:33.152415 29496 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:07:33.154220 29496 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:07:33.154304 29496 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:07:33.154462 29496 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:07:33.154533 29496 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:07:33.200472 29496 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.129:36653
I20260811 09:07:33.200565 29715 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.129:36653 every 8 connection(s)
I20260811 09:07:33.201884 29496 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439250192320-29290-0/minicluster-data/ts-0/data/info.pb
I20260811 09:07:33.212023 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 29496
I20260811 09:07:33.212246 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439250192320-29290-0/minicluster-data/ts-0/wal/instance
I20260811 09:07:33.213225 29290 external_mini_cluster.cc:1428] Setting key 2f548df7c8b4e32d71ae3f9c00e8467d
Aug 11 09:07:33 dist-test-slave-xt4k krb5kdc[29302](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439253, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.129@KRBTEST.COM for kudu/127.28.154.190@KRBTEST.COM
I20260811 09:07:33.229187 29719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:33.205702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.129:57127 (local address 127.28.154.190:34837)
0811 09:07:33.206168 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:33.206173 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:33.207215 (+ 1042us) server_negotiation.cc:407] Connection header received
0811 09:07:33.209261 (+ 2046us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:33.209267 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:33.209372 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:33.209475 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:33.211349 (+ 1874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:33.212200 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:33.214742 (+ 2542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:33.215455 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:33.215599 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:33.220270 (+ 4671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:07:33.220277 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:07:33.220281 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:07:33.220325 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:07:33.224533 (+ 4208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:33.225738 (+ 1205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:33.225743 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:33.225746 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:33.225928 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:33.226468 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:33.226472 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:33.226474 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:33.226864 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:07:33.226993 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:33.228665 (+ 1672us) server_negotiation.cc:299] Negotiation successful
0811 09:07:33.228904 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":242,"threads_started":1}
I20260811 09:07:33.230823 29720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:33.206271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:34837 (local address 127.28.154.129:57127)
0811 09:07:33.207038 (+ 767us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:33.207082 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 09:07:33.208942 (+ 1860us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:07:33.209751 (+ 809us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:07:33.209760 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:07:33.210443 (+ 683us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:07:33.211144 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:33.211167 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:33.212394 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:33.212398 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:33.214576 (+ 2178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:33.214591 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:33.215707 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:33.215717 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:33.216107 (+ 390us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:33.216948 (+ 841us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:07:33.216990 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:07:33.219981 (+ 2991us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:07:33.224813 (+ 4832us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:07:33.224825 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:07:33.224848 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:07:33.225525 (+ 677us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:07:33.226055 (+ 530us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:07:33.226061 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:07:33.226064 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:07:33.226333 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:07:33.227007 (+ 674us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:07:33.227016 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:07:33.227799 (+ 783us) client_negotiation.cc:769] Sending connection context
0811 09:07:33.228447 (+ 648us) client_negotiation.cc:240] Negotiation successful
0811 09:07:33.229182 (+ 735us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":513,"thread_start_us":310,"threads_started":1}
I20260811 09:07:33.232990 29716 heartbeater.cc:347] Connected to a master server at 127.28.154.190:34837
I20260811 09:07:33.233497 29716 heartbeater.cc:464] Registering TS with master...
I20260811 09:07:33.234913 29716 heartbeater.cc:511] Master 127.28.154.190:34837 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.28.154.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.154.130@KRBTEST.COM" created.
I20260811 09:07:33.239583 29375 ts_manager.cc:194] Registered new tserver with Master: 6239333fb48242fa8a85c7aa40de0745 (127.28.154.129:36653)
I20260811 09:07:33.242637 29375 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.154.129@KRBTEST.COM'} at 127.28.154.129:57127
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.154.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.130.keytab.
Entry for principal kudu/127.28.154.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.130.keytab.
I20260811 09:07:33.265252 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439250192320-29290-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439250192320-29290-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439250192320-29290-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439250192320-29290-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.154.130:0
--local_ip_for_outbound_sockets=127.28.154.130
--webserver_interface=127.28.154.130
--webserver_port=0
--tserver_master_addrs=127.28.154.190:34837
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.130.keytab
--principal=kudu/127.28.154.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:38387
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:07:33.664040 29729 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:07:33.664567 29729 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:07:33.664968 29729 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:07:33.673239 29729 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:07:33.673388 29729 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:07:33.673640 29729 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.154.130
I20260811 09:07:33.685101 29729 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:38387
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439250192320-29290-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439250192320-29290-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.130.keytab
--principal=kudu/127.28.154.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.154.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439250192320-29290-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.154.130
--webserver_port=0
--tserver_master_addrs=127.28.154.190:34837
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.154.130
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439250192320-29290-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:07:33.686477 29729 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:07:33.687892 29729 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:07:33.703198 29742 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:07:33.703505 29744 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:07:33.703540 29748 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:07:33.705044 29729 server_base.cc:1034] running on GCE node
I20260811 09:07:33.705699 29729 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:07:33.706709 29729 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:07:33.707923 29729 hybrid_clock.cc:648] HybridClock initialized: now 1786439253707871 us; error 42 us; skew 500 ppm
I20260811 09:07:33.708242 29729 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:07:33 dist-test-slave-xt4k krb5kdc[29302](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439253, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:07:33.711815 29729 init.cc:377] Logged in from keytab as kudu/127.28.154.130@KRBTEST.COM (short username kudu)
I20260811 09:07:33.713474 29729 webserver.cc:460] Webserver started at http://127.28.154.130:35657/ using document root <none> and password file <none>
I20260811 09:07:33.714268 29729 fs_manager.cc:359] Metadata directory not provided
I20260811 09:07:33.714470 29729 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:07:33.714804 29729 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:07:33.718362 29729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439250192320-29290-0/minicluster-data/ts-1/data/instance:
uuid: "510108e40a13441596a19e7df3c08421"
format_stamp: "Formatted at 2026-08-11 09:07:33 on dist-test-slave-xt4k"
server_key: "44105308cc48e209f87218c8035c466b"
server_key_iv: "36cecc6248952bf09ee7bf7d59427eb8"
server_key_version: "encryptionkey@0"
I20260811 09:07:33.719235 29729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439250192320-29290-0/minicluster-data/ts-1/wal/instance:
uuid: "510108e40a13441596a19e7df3c08421"
format_stamp: "Formatted at 2026-08-11 09:07:33 on dist-test-slave-xt4k"
server_key: "44105308cc48e209f87218c8035c466b"
server_key_iv: "36cecc6248952bf09ee7bf7d59427eb8"
server_key_version: "encryptionkey@0"
I20260811 09:07:33.725144 29729 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260811 09:07:33.729115 29764 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:07:33.730906 29729 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.003s
I20260811 09:07:33.731129 29729 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439250192320-29290-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439250192320-29290-0/minicluster-data/ts-1/wal
uuid: "510108e40a13441596a19e7df3c08421"
format_stamp: "Formatted at 2026-08-11 09:07:33 on dist-test-slave-xt4k"
server_key: "44105308cc48e209f87218c8035c466b"
server_key_iv: "36cecc6248952bf09ee7bf7d59427eb8"
server_key_version: "encryptionkey@0"
I20260811 09:07:33.731329 29729 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439250192320-29290-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439250192320-29290-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439250192320-29290-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:07:33.747491 29729 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:07:33.752592 29729 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:07:33.752856 29729 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:07:33.753895 29729 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:07:33.755396 29729 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:07:33.755472 29729 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:07:33.755563 29729 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:07:33.755616 29729 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:07:33.796533 29729 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.130:45197
I20260811 09:07:33.796564 29971 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.130:45197 every 8 connection(s)
I20260811 09:07:33.797953 29729 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439250192320-29290-0/minicluster-data/ts-1/data/info.pb
I20260811 09:07:33.807111 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 29729
I20260811 09:07:33.807307 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439250192320-29290-0/minicluster-data/ts-1/wal/instance
I20260811 09:07:33.807756 29290 external_mini_cluster.cc:1428] Setting key 6e3a7922e662c823d25832e229766c41
Aug 11 09:07:33 dist-test-slave-xt4k krb5kdc[29302](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439253, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.130@KRBTEST.COM for kudu/127.28.154.190@KRBTEST.COM
I20260811 09:07:33.820847 29975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:33.800674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.130:57635 (local address 127.28.154.190:34837)
0811 09:07:33.801113 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:33.801118 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:33.802069 (+ 951us) server_negotiation.cc:407] Connection header received
0811 09:07:33.804000 (+ 1931us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:33.804006 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:33.804096 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:33.804203 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:33.805851 (+ 1648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:33.806618 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:33.808679 (+ 2061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:33.809439 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:33.809571 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:33.814394 (+ 4823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:07:33.814403 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:07:33.814408 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:07:33.814445 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:07:33.817645 (+ 3200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:33.818386 (+ 741us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:33.818392 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:33.818395 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:33.818541 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:33.818943 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:33.818947 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:33.818950 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:33.819378 (+ 428us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:07:33.819519 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:33.820256 (+ 737us) server_negotiation.cc:299] Negotiation successful
0811 09:07:33.820502 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":220,"threads_started":1}
I20260811 09:07:33.821777 29977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:33.801184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:34837 (local address 127.28.154.130:57635)
0811 09:07:33.801901 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:33.801937 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 09:07:33.803728 (+ 1791us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:07:33.804459 (+ 731us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:07:33.804467 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:07:33.805042 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:07:33.805669 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:33.805690 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:33.806777 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:33.806782 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:33.808527 (+ 1745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:33.808543 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:33.809828 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:33.809837 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:33.810248 (+ 411us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:33.811029 (+ 781us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:07:33.811059 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:07:33.814104 (+ 3045us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:07:33.817798 (+ 3694us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:07:33.817806 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:07:33.817821 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:07:33.818256 (+ 435us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:07:33.818634 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:07:33.818638 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:07:33.818640 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:07:33.818831 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:07:33.819512 (+ 681us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:07:33.819526 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:07:33.819878 (+ 352us) client_negotiation.cc:769] Sending connection context
0811 09:07:33.820128 (+ 250us) client_negotiation.cc:240] Negotiation successful
0811 09:07:33.820620 (+ 492us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":283,"threads_started":1}
I20260811 09:07:33.823310 29972 heartbeater.cc:347] Connected to a master server at 127.28.154.190:34837
I20260811 09:07:33.823663 29972 heartbeater.cc:464] Registering TS with master...
I20260811 09:07:33.824797 29972 heartbeater.cc:511] Master 127.28.154.190:34837 requested a full tablet report, sending...
I20260811 09:07:33.826500 29375 ts_manager.cc:194] Registered new tserver with Master: 510108e40a13441596a19e7df3c08421 (127.28.154.130:45197)
I20260811 09:07:33.827309 29375 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.154.130@KRBTEST.COM'} at 127.28.154.130:57635
WARNING: no policy specified for kudu/127.28.154.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.154.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.154.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.131.keytab.
Entry for principal kudu/127.28.154.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.131.keytab.
I20260811 09:07:33.851377 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439250192320-29290-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439250192320-29290-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439250192320-29290-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439250192320-29290-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.154.131:0
--local_ip_for_outbound_sockets=127.28.154.131
--webserver_interface=127.28.154.131
--webserver_port=0
--tserver_master_addrs=127.28.154.190:34837
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.131.keytab
--principal=kudu/127.28.154.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:38387
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:07:34.236817 29982 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:07:34.237182 29982 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:07:34.237349 29982 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:07:34.245617 29982 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:07:34.245795 29982 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:07:34.245970 29982 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.154.131
I20260811 09:07:34.247099 29716 heartbeater.cc:503] Master 127.28.154.190:34837 was elected leader, sending a full tablet report...
I20260811 09:07:34.256616 29982 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:38387
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439250192320-29290-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439250192320-29290-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.131.keytab
--principal=kudu/127.28.154.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.154.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439250192320-29290-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.154.131
--webserver_port=0
--tserver_master_addrs=127.28.154.190:34837
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.154.131
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439250192320-29290-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:07:34.258009 29982 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:07:34.259366 29982 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:07:34.273823 29992 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:07:34.274169 29996 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:07:34.274191 29991 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:07:34.275094 29982 server_base.cc:1034] running on GCE node
I20260811 09:07:34.275749 29982 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:07:34.277174 29982 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:07:34.278447 29982 hybrid_clock.cc:648] HybridClock initialized: now 1786439254278385 us; error 35 us; skew 500 ppm
I20260811 09:07:34.278786 29982 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:07:34 dist-test-slave-xt4k krb5kdc[29302](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439254, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:07:34.282261 29982 init.cc:377] Logged in from keytab as kudu/127.28.154.131@KRBTEST.COM (short username kudu)
I20260811 09:07:34.283985 29982 webserver.cc:460] Webserver started at http://127.28.154.131:40167/ using document root <none> and password file <none>
I20260811 09:07:34.284989 29982 fs_manager.cc:359] Metadata directory not provided
I20260811 09:07:34.285184 29982 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:07:34.285519 29982 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:07:34.289155 29982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439250192320-29290-0/minicluster-data/ts-2/data/instance:
uuid: "7c5b18043904400abb50ec29e129762d"
format_stamp: "Formatted at 2026-08-11 09:07:34 on dist-test-slave-xt4k"
server_key: "1e30006e28d60ad56c0d7e58422cc83d"
server_key_iv: "f07f1c22364f20f76a4d72c545370f2e"
server_key_version: "encryptionkey@0"
I20260811 09:07:34.290045 29982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439250192320-29290-0/minicluster-data/ts-2/wal/instance:
uuid: "7c5b18043904400abb50ec29e129762d"
format_stamp: "Formatted at 2026-08-11 09:07:34 on dist-test-slave-xt4k"
server_key: "1e30006e28d60ad56c0d7e58422cc83d"
server_key_iv: "f07f1c22364f20f76a4d72c545370f2e"
server_key_version: "encryptionkey@0"
I20260811 09:07:34.297335 29982 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.002s sys 0.005s
I20260811 09:07:34.301466 30008 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:07:34.303414 29982 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 09:07:34.303681 29982 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439250192320-29290-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439250192320-29290-0/minicluster-data/ts-2/wal
uuid: "7c5b18043904400abb50ec29e129762d"
format_stamp: "Formatted at 2026-08-11 09:07:34 on dist-test-slave-xt4k"
server_key: "1e30006e28d60ad56c0d7e58422cc83d"
server_key_iv: "f07f1c22364f20f76a4d72c545370f2e"
server_key_version: "encryptionkey@0"
I20260811 09:07:34.303947 29982 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439250192320-29290-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439250192320-29290-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439250192320-29290-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:07:34.330206 29982 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:07:34.335628 29982 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:07:34.335939 29982 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:07:34.337113 29982 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:07:34.338732 29982 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:07:34.338811 29982 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:07:34.338915 29982 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:07:34.338966 29982 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:07:34.380064 29982 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.131:42673
I20260811 09:07:34.380189 30221 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.131:42673 every 8 connection(s)
I20260811 09:07:34.381426 29982 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439250192320-29290-0/minicluster-data/ts-2/data/info.pb
I20260811 09:07:34.382481 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 29982
I20260811 09:07:34.382685 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786439250192320-29290-0/minicluster-data/ts-2/wal/instance
I20260811 09:07:34.383170 29290 external_mini_cluster.cc:1428] Setting key 341a2a4402fc20ff462754726806e217
Aug 11 09:07:34 dist-test-slave-xt4k krb5kdc[29302](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439254, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.131@KRBTEST.COM for kudu/127.28.154.190@KRBTEST.COM
I20260811 09:07:34.403923 30224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:34.384416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.131:33137 (local address 127.28.154.190:34837)
0811 09:07:34.384899 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:34.384905 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:34.386089 (+ 1184us) server_negotiation.cc:407] Connection header received
0811 09:07:34.388646 (+ 2557us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:34.388652 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:34.388744 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:34.388855 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:34.390446 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:34.391253 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:34.392965 (+ 1712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:34.393398 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:34.393512 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:34.397410 (+ 3898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:07:34.397415 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:07:34.397419 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:07:34.397454 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:07:34.400522 (+ 3068us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:34.401394 (+ 872us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:34.401399 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:34.401402 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:34.401610 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:34.402018 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:34.402023 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:34.402027 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:34.402427 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:07:34.402572 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:34.403395 (+ 823us) server_negotiation.cc:299] Negotiation successful
0811 09:07:34.403601 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":207,"threads_started":1}
I20260811 09:07:34.404934 30225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:34.385064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:34837 (local address 127.28.154.131:33137)
0811 09:07:34.385925 (+ 861us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:34.385963 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 09:07:34.388313 (+ 2350us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:07:34.389123 (+ 810us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:07:34.389131 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:07:34.389678 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:07:34.390284 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:34.390301 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:34.391405 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:34.391409 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:34.392825 (+ 1416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:34.392838 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:34.393664 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:34.393674 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:34.393913 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:34.394590 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:07:34.394615 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:07:34.397179 (+ 2564us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:07:34.400656 (+ 3477us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:07:34.400665 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:07:34.400679 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:07:34.401206 (+ 527us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:07:34.401700 (+ 494us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:07:34.401703 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:07:34.401706 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:07:34.401890 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:07:34.402571 (+ 681us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:07:34.402580 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:07:34.402979 (+ 399us) client_negotiation.cc:769] Sending connection context
0811 09:07:34.403244 (+ 265us) client_negotiation.cc:240] Negotiation successful
0811 09:07:34.403675 (+ 431us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":592,"thread_start_us":357,"threads_started":1}
I20260811 09:07:34.406344 30222 heartbeater.cc:347] Connected to a master server at 127.28.154.190:34837
I20260811 09:07:34.406666 30222 heartbeater.cc:464] Registering TS with master...
I20260811 09:07:34.407701 30222 heartbeater.cc:511] Master 127.28.154.190:34837 requested a full tablet report, sending...
I20260811 09:07:34.409373 29375 ts_manager.cc:194] Registered new tserver with Master: 7c5b18043904400abb50ec29e129762d (127.28.154.131:42673)
I20260811 09:07:34.410296 29375 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.154.131@KRBTEST.COM'} at 127.28.154.131:33137
I20260811 09:07:34.413256 29290 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 09:07:34 dist-test-slave-xt4k krb5kdc[29302](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439254, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 09:07:34 dist-test-slave-xt4k krb5kdc[29302](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439254, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.28.154.129@KRBTEST.COM
I20260811 09:07:34.465517 30232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:34.449396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34372 (local address 127.28.154.129:36653)
0811 09:07:34.450189 (+ 793us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:34.450197 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:34.450225 (+ 28us) server_negotiation.cc:407] Connection header received
0811 09:07:34.450289 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:34.450295 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:34.450555 (+ 260us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:34.450689 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:34.452141 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:34.453313 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:34.454795 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:34.455399 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:34.455504 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:34.459927 (+ 4423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:07:34.459942 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:07:34.459960 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:07:34.460000 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:07:34.463007 (+ 3007us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:34.463653 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:34.463660 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:34.463662 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:34.463785 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:34.464112 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:34.464115 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:34.464116 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:34.464520 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:07:34.464799 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:34.465007 (+ 208us) server_negotiation.cc:299] Negotiation successful
0811 09:07:34.465212 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":635,"thread_start_us":470,"threads_started":1}
W20260811 09:07:34.467072 29561 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34372
Aug 11 09:07:34 dist-test-slave-xt4k krb5kdc[29302](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439254, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 09:07:34 dist-test-slave-xt4k krb5kdc[29302](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439254, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.154.129@KRBTEST.COM
I20260811 09:07:34.516865 30232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:34.502030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34386 (local address 127.28.154.129:36653)
0811 09:07:34.502250 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:34.502257 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:34.502411 (+ 154us) server_negotiation.cc:407] Connection header received
0811 09:07:34.502622 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:34.502627 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:34.502732 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:34.502817 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:34.504561 (+ 1744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:34.505297 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:34.506669 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:34.507110 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:34.507206 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:34.511433 (+ 4227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:07:34.511438 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:07:34.511442 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:07:34.511478 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:07:34.514283 (+ 2805us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:34.514959 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:34.514964 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:34.514966 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:34.515121 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:34.515592 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:34.515599 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:34.515601 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:34.515928 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:07:34.516110 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:34.516417 (+ 307us) server_negotiation.cc:299] Negotiation successful
0811 09:07:34.516591 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 09:07:34.518865 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 29496
I20260811 09:07:34.536085 30232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:34.519983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34398 (local address 127.28.154.129:36653)
0811 09:07:34.520136 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:34.520141 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:34.520423 (+ 282us) server_negotiation.cc:407] Connection header received
0811 09:07:34.520582 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:34.520585 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:34.520642 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:34.520727 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:34.521943 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:34.522783 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:34.524160 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:34.524871 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:34.524983 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:34.530222 (+ 5239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:07:34.530228 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:07:34.530231 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:07:34.530269 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:07:34.533252 (+ 2983us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:34.534012 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:34.534017 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:34.534020 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:34.534174 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:34.534624 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:34.534629 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:34.534631 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:34.534946 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:07:34.535223 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:34.535581 (+ 358us) server_negotiation.cc:299] Negotiation successful
0811 09:07:34.535772 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 09:07:34.537060 29707 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:07:34.628682 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 29496
I20260811 09:07:34.655910 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 29729
Aug 11 09:07:34 dist-test-slave-xt4k krb5kdc[29302](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439254, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.154.130@KRBTEST.COM
I20260811 09:07:34.673904 30250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:34.657207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45290 (local address 127.28.154.130:45197)
0811 09:07:34.657901 (+ 694us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:34.657908 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:34.657936 (+ 28us) server_negotiation.cc:407] Connection header received
0811 09:07:34.657999 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:34.658005 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:34.658240 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:34.658387 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:34.660027 (+ 1640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:34.661215 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:34.662644 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:34.663256 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:34.663364 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:34.667725 (+ 4361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:07:34.667769 (+ 44us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:07:34.667794 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:07:34.667848 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:07:34.670810 (+ 2962us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:34.671596 (+ 786us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:34.671604 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:34.671607 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:34.671756 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:34.672175 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:34.672179 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:34.672181 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:34.672738 (+ 557us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:07:34.672980 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:34.673305 (+ 325us) server_negotiation.cc:299] Negotiation successful
0811 09:07:34.673539 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":540,"thread_start_us":331,"threads_started":1}
I20260811 09:07:34.675395 29965 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:07:34.780781 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 29729
I20260811 09:07:34.804484 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 29982
Aug 11 09:07:34 dist-test-slave-xt4k krb5kdc[29302](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439254, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.154.131@KRBTEST.COM
I20260811 09:07:34.820006 30252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:34.805414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49118 (local address 127.28.154.131:42673)
0811 09:07:34.805919 (+ 505us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:34.805926 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:34.805969 (+ 43us) server_negotiation.cc:407] Connection header received
0811 09:07:34.806053 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:34.806062 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:34.806304 (+ 242us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:34.806441 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:34.807609 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:34.808729 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:34.810051 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:34.810653 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:34.810750 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:34.814706 (+ 3956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:07:34.814717 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:07:34.814734 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:07:34.814768 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:07:34.817529 (+ 2761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:34.818167 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:34.818174 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:34.818177 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:34.818314 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:34.818604 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:34.818607 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:34.818609 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:34.818952 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:07:34.819195 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:34.819408 (+ 213us) server_negotiation.cc:299] Negotiation successful
0811 09:07:34.819617 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":224,"threads_started":1}
I20260811 09:07:34.821347 30210 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:07:34.903663 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 29982
I20260811 09:07:34.929942 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 29324
I20260811 09:07:34.931165 29435 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:07:35.038197 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 29324
2026-08-11T09:07:35Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (4743 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439250192320-29290-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:07:35 dist-test-slave-xt4k krb5kdc[30260](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:07:35 dist-test-slave-xt4k krb5kdc[30260](info): set up 2 sockets
Aug 11 09:07:35 dist-test-slave-xt4k krb5kdc[30260](info): commencing operation
krb5kdc: starting...
W20260811 09:07:36.305758 29290 mini_kdc.cc:121] Time spent starting KDC: real 1.218s user 0.000s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439250192320-29290-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439250192320-29290-0/krb5kdc/test-user.keytab.
Aug 11 09:07:36 dist-test-slave-xt4k krb5kdc[30260](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439256, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:07:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:07:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.154.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.154.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.154.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab.
Entry for principal kudu/127.28.154.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab.
I20260811 09:07:36.504282 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439250192320-29290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439250192320-29290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439250192320-29290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.154.190:34395
--webserver_interface=127.28.154.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab
--principal=kudu/127.28.154.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:39775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.154.190:34395
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:07:36.833271 30286 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:07:36.833588 30286 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:07:36.833688 30286 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:07:36.840966 30286 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:07:36.841082 30286 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:07:36.841137 30286 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:07:36.841219 30286 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:07:36.841279 30286 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:07:36.851204 30286 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:39775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439250192320-29290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439250192320-29290-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.154.190:34395
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab
--principal=kudu/127.28.154.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.154.190:34395
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.154.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439250192320-29290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:07:36.852589 30286 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:07:36.853874 30286 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:07:36.865456 30294 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:07:36.865509 30295 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:07:36.865528 30298 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:07:36.866286 30286 server_base.cc:1034] running on GCE node
I20260811 09:07:36.867095 30286 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:07:36.868544 30286 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:07:36.869767 30286 hybrid_clock.cc:648] HybridClock initialized: now 1786439256869721 us; error 34 us; skew 500 ppm
I20260811 09:07:36.870092 30286 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:07:36 dist-test-slave-xt4k krb5kdc[30260](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439256, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:07:36.876286 30286 init.cc:377] Logged in from keytab as kudu/127.28.154.190@KRBTEST.COM (short username kudu)
I20260811 09:07:36.877971 30286 webserver.cc:460] Webserver started at http://127.28.154.190:46577/ using document root <none> and password file <none>
I20260811 09:07:36.878683 30286 fs_manager.cc:359] Metadata directory not provided
I20260811 09:07:36.878789 30286 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:07:36.879074 30286 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:07:36.882222 30286 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439250192320-29290-0/minicluster-data/master-0/data/instance:
uuid: "975b632b1c6f40e1a96d7ac89bd63284"
format_stamp: "Formatted at 2026-08-11 09:07:36 on dist-test-slave-xt4k"
server_key: "e43be111be972841fa105981fd1a2951"
server_key_iv: "729c4ec1a9ce575e88a01f003bec4b55"
server_key_version: "encryptionkey@0"
I20260811 09:07:36.883013 30286 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439250192320-29290-0/minicluster-data/master-0/wal/instance:
uuid: "975b632b1c6f40e1a96d7ac89bd63284"
format_stamp: "Formatted at 2026-08-11 09:07:36 on dist-test-slave-xt4k"
server_key: "e43be111be972841fa105981fd1a2951"
server_key_iv: "729c4ec1a9ce575e88a01f003bec4b55"
server_key_version: "encryptionkey@0"
I20260811 09:07:36.888638 30286 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260811 09:07:36.892194 30312 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:07:36.893594 30286 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 09:07:36.893752 30286 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439250192320-29290-0/minicluster-data/master-0/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439250192320-29290-0/minicluster-data/master-0/wal
uuid: "975b632b1c6f40e1a96d7ac89bd63284"
format_stamp: "Formatted at 2026-08-11 09:07:36 on dist-test-slave-xt4k"
server_key: "e43be111be972841fa105981fd1a2951"
server_key_iv: "729c4ec1a9ce575e88a01f003bec4b55"
server_key_version: "encryptionkey@0"
I20260811 09:07:36.893929 30286 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439250192320-29290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439250192320-29290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439250192320-29290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:07:36.911104 30286 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:07:36.915225 30286 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:07:36.915473 30286 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:07:36.932929 30286 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.190:34395
I20260811 09:07:36.932986 30426 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.190:34395 every 8 connection(s)
I20260811 09:07:36.934314 30286 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
I20260811 09:07:36.938054 30427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:07:36.943472 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 30286
I20260811 09:07:36.943713 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439250192320-29290-0/minicluster-data/master-0/wal/instance
I20260811 09:07:36.944555 29290 external_mini_cluster.cc:1428] Setting key ce11cb3b94bd026bd03a73abd730037b
I20260811 09:07:36.949697 30427 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 975b632b1c6f40e1a96d7ac89bd63284: Bootstrap starting.
I20260811 09:07:36.953409 30427 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 975b632b1c6f40e1a96d7ac89bd63284: Neither blocks nor log segments found. Creating new log.
I20260811 09:07:36.954413 30427 log.cc:824] T 00000000000000000000000000000000 P 975b632b1c6f40e1a96d7ac89bd63284: Log is configured to *not* fsync() on all Append() calls
I20260811 09:07:36.957656 30427 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 975b632b1c6f40e1a96d7ac89bd63284: No bootstrap required, opened a new log
Aug 11 09:07:36 dist-test-slave-xt4k krb5kdc[30260](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439256, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.154.190@KRBTEST.COM
I20260811 09:07:36.966706 30427 raft_consensus.cc:348] T 00000000000000000000000000000000 P 975b632b1c6f40e1a96d7ac89bd63284 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "975b632b1c6f40e1a96d7ac89bd63284" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 34395 } }
I20260811 09:07:36.967104 30427 raft_consensus.cc:374] T 00000000000000000000000000000000 P 975b632b1c6f40e1a96d7ac89bd63284 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:07:36.967232 30427 raft_consensus.cc:729] T 00000000000000000000000000000000 P 975b632b1c6f40e1a96d7ac89bd63284 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 975b632b1c6f40e1a96d7ac89bd63284, State: Initialized, Role: FOLLOWER
I20260811 09:07:36.967965 30427 consensus_queue.cc:260] T 00000000000000000000000000000000 P 975b632b1c6f40e1a96d7ac89bd63284 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "975b632b1c6f40e1a96d7ac89bd63284" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 34395 } }
I20260811 09:07:36.968266 30427 raft_consensus.cc:388] T 00000000000000000000000000000000 P 975b632b1c6f40e1a96d7ac89bd63284 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:07:36.968434 30427 raft_consensus.cc:482] T 00000000000000000000000000000000 P 975b632b1c6f40e1a96d7ac89bd63284 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:07:36.968612 30427 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 975b632b1c6f40e1a96d7ac89bd63284 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:07:36.968847 30439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:36.946759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55824 (local address 127.28.154.190:34395)
0811 09:07:36.947585 (+ 826us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:36.947602 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:36.947652 (+ 50us) server_negotiation.cc:407] Connection header received
0811 09:07:36.949420 (+ 1768us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:36.949446 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:36.949889 (+ 443us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:36.950244 (+ 355us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:36.952097 (+ 1853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:36.953480 (+ 1383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:36.954772 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:36.955868 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:36.956062 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:36.960106 (+ 4044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:07:36.960115 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:07:36.960132 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:07:36.960178 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:07:36.964590 (+ 4412us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:36.965318 (+ 728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:36.965329 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:36.965338 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:36.965511 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:36.965893 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:36.965898 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:36.965902 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:36.966485 (+ 583us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:07:36.966751 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:36.967260 (+ 509us) server_negotiation.cc:299] Negotiation successful
0811 09:07:36.967683 (+ 423us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":602,"thread_start_us":364,"threads_started":1}
I20260811 09:07:36.971433 30427 raft_consensus.cc:504] T 00000000000000000000000000000000 P 975b632b1c6f40e1a96d7ac89bd63284 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "975b632b1c6f40e1a96d7ac89bd63284" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 34395 } }
I20260811 09:07:36.972007 30427 leader_election.cc:302] T 00000000000000000000000000000000 P 975b632b1c6f40e1a96d7ac89bd63284 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 975b632b1c6f40e1a96d7ac89bd63284; no voters:
I20260811 09:07:36.972684 30427 leader_election.cc:288] T 00000000000000000000000000000000 P 975b632b1c6f40e1a96d7ac89bd63284 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:07:36.972790 30442 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 975b632b1c6f40e1a96d7ac89bd63284 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:07:36.973419 30442 raft_consensus.cc:686] T 00000000000000000000000000000000 P 975b632b1c6f40e1a96d7ac89bd63284 [term 1 LEADER]: Becoming Leader. State: Replica: 975b632b1c6f40e1a96d7ac89bd63284, State: Running, Role: LEADER
I20260811 09:07:36.973948 30442 consensus_queue.cc:237] T 00000000000000000000000000000000 P 975b632b1c6f40e1a96d7ac89bd63284 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "975b632b1c6f40e1a96d7ac89bd63284" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 34395 } }
I20260811 09:07:36.974684 30427 sys_catalog.cc:564] T 00000000000000000000000000000000 P 975b632b1c6f40e1a96d7ac89bd63284 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:07:36.976439 30445 sys_catalog.cc:455] T 00000000000000000000000000000000 P 975b632b1c6f40e1a96d7ac89bd63284 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 975b632b1c6f40e1a96d7ac89bd63284. Latest consensus state: current_term: 1 leader_uuid: "975b632b1c6f40e1a96d7ac89bd63284" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "975b632b1c6f40e1a96d7ac89bd63284" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 34395 } } }
I20260811 09:07:36.976897 30445 sys_catalog.cc:458] T 00000000000000000000000000000000 P 975b632b1c6f40e1a96d7ac89bd63284 [sys.catalog]: This master's current role is: LEADER
I20260811 09:07:36.977267 30444 sys_catalog.cc:455] T 00000000000000000000000000000000 P 975b632b1c6f40e1a96d7ac89bd63284 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "975b632b1c6f40e1a96d7ac89bd63284" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "975b632b1c6f40e1a96d7ac89bd63284" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 34395 } } }
I20260811 09:07:36.977562 30444 sys_catalog.cc:458] T 00000000000000000000000000000000 P 975b632b1c6f40e1a96d7ac89bd63284 [sys.catalog]: This master's current role is: LEADER
I20260811 09:07:36.980986 30457 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:07:36.986487 30457 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:07:36.995715 30457 catalog_manager.cc:1347] Generated new cluster ID: 9d971e09a10d4b66a7d20aa591f9d450
I20260811 09:07:36.995874 30457 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:07:37.027951 30457 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:07:37.029803 30457 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:07:37.039143 30457 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 975b632b1c6f40e1a96d7ac89bd63284: Generated new TSK 0
I20260811 09:07:37.040097 30457 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.28.154.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.154.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.154.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.129.keytab.
Entry for principal kudu/127.28.154.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.129.keytab.
I20260811 09:07:37.102701 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439250192320-29290-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439250192320-29290-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439250192320-29290-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439250192320-29290-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.154.129:0
--local_ip_for_outbound_sockets=127.28.154.129
--webserver_interface=127.28.154.129
--webserver_port=0
--tserver_master_addrs=127.28.154.190:34395
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.129.keytab
--principal=kudu/127.28.154.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:39775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:07:37.448179 30484 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:07:37.448571 30484 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:07:37.448693 30484 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:07:37.456403 30484 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:07:37.456528 30484 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:07:37.456656 30484 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.154.129
I20260811 09:07:37.467221 30484 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:39775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439250192320-29290-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439250192320-29290-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.129.keytab
--principal=kudu/127.28.154.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.154.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439250192320-29290-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.154.129
--webserver_port=0
--tserver_master_addrs=127.28.154.190:34395
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.154.129
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439250192320-29290-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:07:37.468989 30484 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:07:37.470391 30484 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:07:37.484501 30491 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:07:37.484531 30497 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:07:37.484498 30492 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:07:37.485350 30484 server_base.cc:1034] running on GCE node
I20260811 09:07:37.486058 30484 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:07:37.487035 30484 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:07:37.488256 30484 hybrid_clock.cc:648] HybridClock initialized: now 1786439257488187 us; error 52 us; skew 500 ppm
I20260811 09:07:37.488660 30484 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:07:37 dist-test-slave-xt4k krb5kdc[30260](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439257, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:07:37.491868 30484 init.cc:377] Logged in from keytab as kudu/127.28.154.129@KRBTEST.COM (short username kudu)
I20260811 09:07:37.493335 30484 webserver.cc:460] Webserver started at http://127.28.154.129:46677/ using document root <none> and password file <none>
I20260811 09:07:37.494024 30484 fs_manager.cc:359] Metadata directory not provided
I20260811 09:07:37.494128 30484 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:07:37.494405 30484 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:07:37.497557 30484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439250192320-29290-0/minicluster-data/ts-0/data/instance:
uuid: "55d7238af4e440afa4c0803c769ddb9e"
format_stamp: "Formatted at 2026-08-11 09:07:37 on dist-test-slave-xt4k"
server_key: "3633f65213391f5a3ddf441118e64cac"
server_key_iv: "14e690a6e311fc132e9d6594a6aeb725"
server_key_version: "encryptionkey@0"
I20260811 09:07:37.498359 30484 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439250192320-29290-0/minicluster-data/ts-0/wal/instance:
uuid: "55d7238af4e440afa4c0803c769ddb9e"
format_stamp: "Formatted at 2026-08-11 09:07:37 on dist-test-slave-xt4k"
server_key: "3633f65213391f5a3ddf441118e64cac"
server_key_iv: "14e690a6e311fc132e9d6594a6aeb725"
server_key_version: "encryptionkey@0"
I20260811 09:07:37.503881 30484 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 09:07:37.507849 30507 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:07:37.509574 30484 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 09:07:37.509766 30484 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439250192320-29290-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439250192320-29290-0/minicluster-data/ts-0/wal
uuid: "55d7238af4e440afa4c0803c769ddb9e"
format_stamp: "Formatted at 2026-08-11 09:07:37 on dist-test-slave-xt4k"
server_key: "3633f65213391f5a3ddf441118e64cac"
server_key_iv: "14e690a6e311fc132e9d6594a6aeb725"
server_key_version: "encryptionkey@0"
I20260811 09:07:37.509980 30484 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439250192320-29290-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439250192320-29290-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439250192320-29290-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:07:37.531448 30484 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:07:37.536453 30484 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:07:37.536921 30484 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:07:37.538151 30484 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:07:37.539671 30484 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:07:37.539750 30484 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:07:37.539876 30484 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:07:37.539937 30484 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:07:37.572329 30484 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.129:45879
I20260811 09:07:37.572420 30720 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.129:45879 every 8 connection(s)
I20260811 09:07:37.573721 30484 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439250192320-29290-0/minicluster-data/ts-0/data/info.pb
I20260811 09:07:37.577091 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 30484
I20260811 09:07:37.577292 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439250192320-29290-0/minicluster-data/ts-0/wal/instance
I20260811 09:07:37.577968 29290 external_mini_cluster.cc:1428] Setting key 1c19dc783913357017f56e3b32cc6686
Aug 11 09:07:37 dist-test-slave-xt4k krb5kdc[30260](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439257, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.129@KRBTEST.COM for kudu/127.28.154.190@KRBTEST.COM
I20260811 09:07:37.595909 30724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:37.576835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.129:36149 (local address 127.28.154.190:34395)
0811 09:07:37.577325 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:37.577332 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:37.578389 (+ 1057us) server_negotiation.cc:407] Connection header received
0811 09:07:37.580618 (+ 2229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:37.580622 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:37.580693 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:37.580820 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:37.582850 (+ 2030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:37.583617 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:37.585793 (+ 2176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:37.586395 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:37.586555 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:37.590248 (+ 3693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:07:37.590254 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:07:37.590257 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:07:37.590293 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:07:37.592808 (+ 2515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:37.593556 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:37.593560 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:37.593563 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:37.593685 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:37.594117 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:37.594121 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:37.594123 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:37.594387 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:07:37.594530 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:37.595302 (+ 772us) server_negotiation.cc:299] Negotiation successful
0811 09:07:37.595551 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":241,"threads_started":1}
WARNING: no policy specified for kudu/127.28.154.130@KRBTEST.COM; defaulting to no policy
I20260811 09:07:37.596992 30727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:37.577328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:34395 (local address 127.28.154.129:36149)
0811 09:07:37.578202 (+ 874us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:37.578258 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 09:07:37.580282 (+ 2024us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:07:37.581100 (+ 818us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:07:37.581112 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:07:37.581848 (+ 736us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:07:37.582665 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:37.582691 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:37.583828 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:37.583833 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:37.585638 (+ 1805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:37.585655 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:37.586685 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:37.586691 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:37.586905 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:37.587605 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:07:37.587644 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:07:37.590015 (+ 2371us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:07:37.592944 (+ 2929us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:07:37.592962 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:07:37.592983 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:07:37.593427 (+ 444us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:07:37.593791 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:07:37.593796 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:07:37.593800 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:07:37.594001 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:07:37.594515 (+ 514us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:07:37.594524 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:07:37.594889 (+ 365us) client_negotiation.cc:769] Sending connection context
0811 09:07:37.595150 (+ 261us) client_negotiation.cc:240] Negotiation successful
0811 09:07:37.595697 (+ 547us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":531,"thread_start_us":297,"threads_started":1}
I20260811 09:07:37.598722 30722 heartbeater.cc:347] Connected to a master server at 127.28.154.190:34395
I20260811 09:07:37.599033 30722 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.154.130@KRBTEST.COM" created.
I20260811 09:07:37.599797 30722 heartbeater.cc:511] Master 127.28.154.190:34395 requested a full tablet report, sending...
I20260811 09:07:37.602025 30347 ts_manager.cc:194] Registered new tserver with Master: 55d7238af4e440afa4c0803c769ddb9e (127.28.154.129:45879)
I20260811 09:07:37.603587 30347 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.154.129@KRBTEST.COM'} at 127.28.154.129:36149
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.154.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.130.keytab.
Entry for principal kudu/127.28.154.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.130.keytab.
I20260811 09:07:37.621642 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439250192320-29290-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439250192320-29290-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439250192320-29290-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439250192320-29290-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.154.130:0
--local_ip_for_outbound_sockets=127.28.154.130
--webserver_interface=127.28.154.130
--webserver_port=0
--tserver_master_addrs=127.28.154.190:34395
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.130.keytab
--principal=kudu/127.28.154.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:39775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:07:37.954622 30731 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:07:37.954931 30731 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:07:37.955062 30731 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:07:37.962740 30731 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:07:37.962858 30731 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:07:37.963012 30731 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.154.130
I20260811 09:07:37.974288 30731 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:39775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439250192320-29290-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439250192320-29290-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.130.keytab
--principal=kudu/127.28.154.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.154.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439250192320-29290-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.154.130
--webserver_port=0
--tserver_master_addrs=127.28.154.190:34395
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.154.130
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439250192320-29290-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:07:37.975721 30731 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:07:37.977149 30731 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:07:37.990698 30739 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:07:37.990665 30740 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:07:37.991050 30742 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:07:37.991496 30731 server_base.cc:1034] running on GCE node
I20260811 09:07:37.992192 30731 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:07:37.993278 30731 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:07:37.994524 30731 hybrid_clock.cc:648] HybridClock initialized: now 1786439257994435 us; error 79 us; skew 500 ppm
I20260811 09:07:37.994989 30731 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:07:37 dist-test-slave-xt4k krb5kdc[30260](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439257, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:07:37.998725 30731 init.cc:377] Logged in from keytab as kudu/127.28.154.130@KRBTEST.COM (short username kudu)
I20260811 09:07:38.000406 30731 webserver.cc:460] Webserver started at http://127.28.154.130:44085/ using document root <none> and password file <none>
I20260811 09:07:38.001365 30731 fs_manager.cc:359] Metadata directory not provided
I20260811 09:07:38.001581 30731 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:07:38.001948 30731 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:07:38.005576 30731 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439250192320-29290-0/minicluster-data/ts-1/data/instance:
uuid: "6faa80acac7144b7965315c270d55ccc"
format_stamp: "Formatted at 2026-08-11 09:07:37 on dist-test-slave-xt4k"
server_key: "62c5054d0441deec5d5048e22431107c"
server_key_iv: "e2f93e6c224aa511c29d58fdd9054152"
server_key_version: "encryptionkey@0"
I20260811 09:07:38.006481 30731 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439250192320-29290-0/minicluster-data/ts-1/wal/instance:
uuid: "6faa80acac7144b7965315c270d55ccc"
format_stamp: "Formatted at 2026-08-11 09:07:37 on dist-test-slave-xt4k"
server_key: "62c5054d0441deec5d5048e22431107c"
server_key_iv: "e2f93e6c224aa511c29d58fdd9054152"
server_key_version: "encryptionkey@0"
I20260811 09:07:38.012864 30731 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260811 09:07:38.017165 30754 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:07:38.018882 30731 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 09:07:38.019080 30731 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439250192320-29290-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439250192320-29290-0/minicluster-data/ts-1/wal
uuid: "6faa80acac7144b7965315c270d55ccc"
format_stamp: "Formatted at 2026-08-11 09:07:37 on dist-test-slave-xt4k"
server_key: "62c5054d0441deec5d5048e22431107c"
server_key_iv: "e2f93e6c224aa511c29d58fdd9054152"
server_key_version: "encryptionkey@0"
I20260811 09:07:38.019309 30731 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439250192320-29290-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439250192320-29290-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439250192320-29290-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:07:38.038031 30731 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:07:38.042636 30731 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:07:38.043017 30731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:07:38.044062 30731 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:07:38.045550 30731 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:07:38.045696 30731 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:07:38.045878 30731 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:07:38.045941 30731 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:07:38.085263 30731 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.130:43949
I20260811 09:07:38.085316 30971 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.130:43949 every 8 connection(s)
I20260811 09:07:38.086607 30731 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439250192320-29290-0/minicluster-data/ts-1/data/info.pb
I20260811 09:07:38.091856 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 30731
I20260811 09:07:38.092077 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439250192320-29290-0/minicluster-data/ts-1/wal/instance
I20260811 09:07:38.092703 29290 external_mini_cluster.cc:1428] Setting key 48ef2f672e6bf4c6777a62c80e1b3a56
Aug 11 09:07:38 dist-test-slave-xt4k krb5kdc[30260](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439257, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.130@KRBTEST.COM for kudu/127.28.154.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.28.154.131@KRBTEST.COM; defaulting to no policy
I20260811 09:07:38.110827 30724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:38.090078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.130:35199 (local address 127.28.154.190:34395)
0811 09:07:38.090306 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:38.090312 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:38.091838 (+ 1526us) server_negotiation.cc:407] Connection header received
0811 09:07:38.094357 (+ 2519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:38.094363 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:38.094441 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:38.094543 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:38.096411 (+ 1868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:38.097246 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:38.099640 (+ 2394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:38.100243 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:38.100423 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:38.104519 (+ 4096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:07:38.104550 (+ 31us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:07:38.104554 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:07:38.104590 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:07:38.107338 (+ 2748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:38.108252 (+ 914us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:38.108258 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:38.108262 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:38.108434 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:38.108886 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:38.108892 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:38.108895 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:38.109209 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:07:38.109351 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:38.110202 (+ 851us) server_negotiation.cc:299] Negotiation successful
0811 09:07:38.110461 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260811 09:07:38.111990 30975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:38.090698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:34395 (local address 127.28.154.130:35199)
0811 09:07:38.091579 (+ 881us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:38.091682 (+ 103us) client_negotiation.cc:174] Beginning negotiation
0811 09:07:38.094002 (+ 2320us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:07:38.094912 (+ 910us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:07:38.094926 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:07:38.095550 (+ 624us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:07:38.096198 (+ 648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:38.096219 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:38.097423 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:38.097429 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:38.099482 (+ 2053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:38.099497 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:38.100544 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:38.100552 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:38.100793 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:38.101506 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:07:38.101540 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:07:38.104266 (+ 2726us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:07:38.107497 (+ 3231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:07:38.107515 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:07:38.107534 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:07:38.108099 (+ 565us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:07:38.108549 (+ 450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:07:38.108554 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:07:38.108558 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:07:38.108772 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:07:38.109352 (+ 580us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:07:38.109360 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:07:38.109770 (+ 410us) client_negotiation.cc:769] Sending connection context
0811 09:07:38.110037 (+ 267us) client_negotiation.cc:240] Negotiation successful
0811 09:07:38.110667 (+ 630us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":582,"thread_start_us":348,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.154.131@KRBTEST.COM" created.
I20260811 09:07:38.113633 30972 heartbeater.cc:347] Connected to a master server at 127.28.154.190:34395
I20260811 09:07:38.113986 30972 heartbeater.cc:464] Registering TS with master...
I20260811 09:07:38.114889 30972 heartbeater.cc:511] Master 127.28.154.190:34395 requested a full tablet report, sending...
I20260811 09:07:38.116650 30347 ts_manager.cc:194] Registered new tserver with Master: 6faa80acac7144b7965315c270d55ccc (127.28.154.130:43949)
I20260811 09:07:38.117648 30347 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.154.130@KRBTEST.COM'} at 127.28.154.130:35199
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.154.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.131.keytab.
Entry for principal kudu/127.28.154.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.131.keytab.
I20260811 09:07:38.139516 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439250192320-29290-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439250192320-29290-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439250192320-29290-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439250192320-29290-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.154.131:0
--local_ip_for_outbound_sockets=127.28.154.131
--webserver_interface=127.28.154.131
--webserver_port=0
--tserver_master_addrs=127.28.154.190:34395
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.131.keytab
--principal=kudu/127.28.154.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:39775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:07:38.520323 30978 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:07:38.520673 30978 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:07:38.520782 30978 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:07:38.528646 30978 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:07:38.528766 30978 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:07:38.528903 30978 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.154.131
I20260811 09:07:38.539271 30978 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:39775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439250192320-29290-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439250192320-29290-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.131.keytab
--principal=kudu/127.28.154.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.154.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439250192320-29290-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.154.131
--webserver_port=0
--tserver_master_addrs=127.28.154.190:34395
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.154.131
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439250192320-29290-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:07:38.540634 30978 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:07:38.542184 30978 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:07:38.557229 30986 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:07:38.557277 30988 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:07:38.557253 30994 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:07:38.558264 30978 server_base.cc:1034] running on GCE node
I20260811 09:07:38.558943 30978 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:07:38.559923 30978 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:07:38.561141 30978 hybrid_clock.cc:648] HybridClock initialized: now 1786439258561096 us; error 49 us; skew 500 ppm
I20260811 09:07:38.561460 30978 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:07:38 dist-test-slave-xt4k krb5kdc[30260](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439258, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:07:38.564962 30978 init.cc:377] Logged in from keytab as kudu/127.28.154.131@KRBTEST.COM (short username kudu)
I20260811 09:07:38.566433 30978 webserver.cc:460] Webserver started at http://127.28.154.131:46881/ using document root <none> and password file <none>
I20260811 09:07:38.567134 30978 fs_manager.cc:359] Metadata directory not provided
I20260811 09:07:38.567253 30978 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:07:38.567543 30978 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:07:38.571067 30978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439250192320-29290-0/minicluster-data/ts-2/data/instance:
uuid: "74fb90795ebb461aa84b8b6d1d9b2851"
format_stamp: "Formatted at 2026-08-11 09:07:38 on dist-test-slave-xt4k"
server_key: "f0b46a38124e7f7496857f80c704e287"
server_key_iv: "d8b8ab105b7f6e6384641e5d6524d1ee"
server_key_version: "encryptionkey@0"
I20260811 09:07:38.572136 30978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439250192320-29290-0/minicluster-data/ts-2/wal/instance:
uuid: "74fb90795ebb461aa84b8b6d1d9b2851"
format_stamp: "Formatted at 2026-08-11 09:07:38 on dist-test-slave-xt4k"
server_key: "f0b46a38124e7f7496857f80c704e287"
server_key_iv: "d8b8ab105b7f6e6384641e5d6524d1ee"
server_key_version: "encryptionkey@0"
I20260811 09:07:38.579231 30978 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.001s
I20260811 09:07:38.583285 31004 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:07:38.584887 30978 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 09:07:38.585135 30978 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439250192320-29290-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439250192320-29290-0/minicluster-data/ts-2/wal
uuid: "74fb90795ebb461aa84b8b6d1d9b2851"
format_stamp: "Formatted at 2026-08-11 09:07:38 on dist-test-slave-xt4k"
server_key: "f0b46a38124e7f7496857f80c704e287"
server_key_iv: "d8b8ab105b7f6e6384641e5d6524d1ee"
server_key_version: "encryptionkey@0"
I20260811 09:07:38.585443 30978 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439250192320-29290-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439250192320-29290-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439250192320-29290-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:07:38.602959 30978 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:07:38.606575 30722 heartbeater.cc:503] Master 127.28.154.190:34395 was elected leader, sending a full tablet report...
I20260811 09:07:38.607635 30978 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:07:38.608057 30978 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:07:38.609591 30978 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:07:38.611182 30978 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:07:38.611266 30978 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:07:38.611394 30978 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:07:38.611455 30978 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:07:38.653041 30978 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.131:44967
I20260811 09:07:38.653112 31223 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.131:44967 every 8 connection(s)
I20260811 09:07:38.654440 30978 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439250192320-29290-0/minicluster-data/ts-2/data/info.pb
I20260811 09:07:38.661055 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 30978
I20260811 09:07:38.661248 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786439250192320-29290-0/minicluster-data/ts-2/wal/instance
I20260811 09:07:38.661803 29290 external_mini_cluster.cc:1428] Setting key da9e40123864555ebcaf55aaed2ec8ad
Aug 11 09:07:38 dist-test-slave-xt4k krb5kdc[30260](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439258, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.131@KRBTEST.COM for kudu/127.28.154.190@KRBTEST.COM
I20260811 09:07:38.679597 31227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:38.657451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.131:53727 (local address 127.28.154.190:34395)
0811 09:07:38.658082 (+ 631us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:38.658087 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:38.659085 (+ 998us) server_negotiation.cc:407] Connection header received
0811 09:07:38.661156 (+ 2071us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:38.661162 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:38.661261 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:38.661373 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:38.663639 (+ 2266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:38.664907 (+ 1268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:38.666885 (+ 1978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:38.667602 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:38.667737 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:38.672755 (+ 5018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:07:38.672760 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:07:38.672764 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:07:38.672802 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:07:38.675773 (+ 2971us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:38.676879 (+ 1106us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:38.676910 (+ 31us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:38.676913 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:38.677081 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:38.677650 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:38.677654 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:38.677656 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:38.678084 (+ 428us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:07:38.678241 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:38.679097 (+ 856us) server_negotiation.cc:299] Negotiation successful
0811 09:07:38.679291 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":498,"thread_start_us":352,"threads_started":1}
I20260811 09:07:38.680758 31231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:38.657974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:34395 (local address 127.28.154.131:53727)
0811 09:07:38.658855 (+ 881us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:38.658912 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 09:07:38.660840 (+ 1928us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:07:38.661717 (+ 877us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:07:38.661730 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:07:38.662577 (+ 847us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:07:38.663437 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:38.663462 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:38.665084 (+ 1622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:38.665090 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:38.666596 (+ 1506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:38.666610 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:38.667991 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:38.667999 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:38.668329 (+ 330us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:38.669454 (+ 1125us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:07:38.669485 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:07:38.672457 (+ 2972us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:07:38.675983 (+ 3526us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:07:38.675995 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:07:38.676016 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:07:38.676747 (+ 731us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:07:38.677204 (+ 457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:07:38.677209 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:07:38.677213 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:07:38.677496 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:07:38.678221 (+ 725us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:07:38.678230 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:07:38.678720 (+ 490us) client_negotiation.cc:769] Sending connection context
0811 09:07:38.678970 (+ 250us) client_negotiation.cc:240] Negotiation successful
0811 09:07:38.679412 (+ 442us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":592,"thread_start_us":312,"threads_started":1}
I20260811 09:07:38.682370 31225 heartbeater.cc:347] Connected to a master server at 127.28.154.190:34395
I20260811 09:07:38.682708 31225 heartbeater.cc:464] Registering TS with master...
I20260811 09:07:38.683633 31225 heartbeater.cc:511] Master 127.28.154.190:34395 requested a full tablet report, sending...
I20260811 09:07:38.685366 30347 ts_manager.cc:194] Registered new tserver with Master: 74fb90795ebb461aa84b8b6d1d9b2851 (127.28.154.131:44967)
I20260811 09:07:38.686244 30347 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.154.131@KRBTEST.COM'} at 127.28.154.131:53727
I20260811 09:07:38.691030 29290 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 09:07:38.713516 31227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:38.699355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55832 (local address 127.28.154.190:34395)
0811 09:07:38.699611 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:38.699618 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:38.700059 (+ 441us) server_negotiation.cc:407] Connection header received
0811 09:07:38.700251 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:38.700255 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:38.700325 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:38.700463 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:38.702156 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:38.702935 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:38.704207 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:38.704956 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:38.705096 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:38.708060 (+ 2964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:07:38.708065 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:07:38.708069 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:07:38.708105 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:07:38.710661 (+ 2556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:38.711443 (+ 782us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:38.711449 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:38.711452 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:38.711597 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:38.711980 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:38.711984 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:38.711986 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:38.712323 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:07:38.712581 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:38.712941 (+ 360us) server_negotiation.cc:299] Negotiation successful
0811 09:07:38.713186 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260811 09:07:38.723229 30347 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55832:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 09:07:38.732756 30347 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 09:07:38.762223 31255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:38.750675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.129:45879 (local address 127.0.0.1:42986)
0811 09:07:38.751488 (+ 813us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:38.751527 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 09:07:38.751769 (+ 242us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:07:38.754354 (+ 2585us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:07:38.754363 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:07:38.754399 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:07:38.754889 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:38.754903 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:38.756738 (+ 1835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:38.756743 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:38.759168 (+ 2425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:38.759184 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:38.761144 (+ 1960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:38.761150 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:38.761352 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:38.761412 (+ 60us) client_negotiation.cc:769] Sending connection context
0811 09:07:38.761549 (+ 137us) client_negotiation.cc:240] Negotiation successful
0811 09:07:38.761755 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":526,"mutex_wait_us":42,"thread_start_us":346,"threads_started":1}
I20260811 09:07:38.762992 31258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:38.751237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42986 (local address 127.28.154.129:45879)
0811 09:07:38.753613 (+ 2376us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:38.753621 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:38.753651 (+ 30us) server_negotiation.cc:407] Connection header received
0811 09:07:38.753751 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:38.753759 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:38.754070 (+ 311us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:38.754948 (+ 878us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:07:38.755165 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:38.756576 (+ 1411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:38.759372 (+ 2796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:38.760931 (+ 1559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:38.762019 (+ 1088us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:38.762205 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:38.762290 (+ 85us) server_negotiation.cc:299] Negotiation successful
0811 09:07:38.762477 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2201,"thread_start_us":2054,"threads_started":1}
I20260811 09:07:38.763480 31257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:38.751257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.130:43949 (local address 127.0.0.1:40992)
0811 09:07:38.752903 (+ 1646us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:38.752923 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 09:07:38.753052 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:07:38.755480 (+ 2428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:07:38.755485 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:07:38.755550 (+ 65us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:07:38.755974 (+ 424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:38.755984 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:38.758272 (+ 2288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:38.758277 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:38.760492 (+ 2215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:38.760508 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:38.762632 (+ 2124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:38.762638 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:38.762813 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:38.762846 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 09:07:38.762927 (+ 81us) client_negotiation.cc:240] Negotiation successful
0811 09:07:38.763062 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1507,"mutex_wait_us":3,"spinlock_wait_cycles":8448,"thread_start_us":907,"threads_started":1}
I20260811 09:07:38.763799 31265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:38.752680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52402 (local address 127.28.154.131:44967)
0811 09:07:38.754033 (+ 1353us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:38.754043 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:38.754889 (+ 846us) server_negotiation.cc:407] Connection header received
0811 09:07:38.754993 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:38.755000 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:38.755222 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:38.755429 (+ 207us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:07:38.756162 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:38.757809 (+ 1647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:38.760258 (+ 2449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:38.761642 (+ 1384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:38.761833 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:38.762016 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:38.763285 (+ 1269us) server_negotiation.cc:299] Negotiation successful
0811 09:07:38.763459 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1159,"thread_start_us":979,"threads_started":1}
I20260811 09:07:38.763096 31264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:38.751938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.131:44967 (local address 127.0.0.1:52402)
0811 09:07:38.754578 (+ 2640us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:38.754600 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 09:07:38.754743 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:07:38.755418 (+ 675us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:07:38.755422 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:07:38.755446 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:07:38.755936 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:38.755952 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:38.757954 (+ 2002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:38.757960 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:38.760110 (+ 2150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:38.760123 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:38.762187 (+ 2064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:38.762196 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:38.762401 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:38.762436 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 09:07:38.762518 (+ 82us) client_negotiation.cc:240] Negotiation successful
0811 09:07:38.762662 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2505,"mutex_wait_us":41,"thread_start_us":2318,"threads_started":1}
I20260811 09:07:38.763993 31259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:38.751846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40992 (local address 127.28.154.130:43949)
0811 09:07:38.754982 (+ 3136us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:38.754989 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:38.755021 (+ 32us) server_negotiation.cc:407] Connection header received
0811 09:07:38.755113 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:38.755120 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:38.755285 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:38.756163 (+ 878us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:07:38.756439 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:38.758137 (+ 1698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:38.760869 (+ 2732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:38.762401 (+ 1532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:38.763230 (+ 829us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:38.763397 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:38.763479 (+ 82us) server_negotiation.cc:299] Negotiation successful
0811 09:07:38.763650 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2987,"thread_start_us":2818,"threads_started":1}
I20260811 09:07:38.767856 30587 tablet_service.cc:1467] Processing CreateTablet for tablet f987040e4cb14fc8af61af2f6571a881 (DEFAULT_TABLE table=test-table [id=be45c47831184543886de2bbf228a893]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:07:38.767910 30853 tablet_service.cc:1467] Processing CreateTablet for tablet f987040e4cb14fc8af61af2f6571a881 (DEFAULT_TABLE table=test-table [id=be45c47831184543886de2bbf228a893]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:07:38.767910 31102 tablet_service.cc:1467] Processing CreateTablet for tablet f987040e4cb14fc8af61af2f6571a881 (DEFAULT_TABLE table=test-table [id=be45c47831184543886de2bbf228a893]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:07:38.769428 30587 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f987040e4cb14fc8af61af2f6571a881. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:07:38.769500 31102 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f987040e4cb14fc8af61af2f6571a881. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:07:38.769536 30853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f987040e4cb14fc8af61af2f6571a881. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:07:38.782760 31266 tablet_bootstrap.cc:492] T f987040e4cb14fc8af61af2f6571a881 P 55d7238af4e440afa4c0803c769ddb9e: Bootstrap starting.
I20260811 09:07:38.784834 31269 tablet_bootstrap.cc:492] T f987040e4cb14fc8af61af2f6571a881 P 74fb90795ebb461aa84b8b6d1d9b2851: Bootstrap starting.
I20260811 09:07:38.785691 31268 tablet_bootstrap.cc:492] T f987040e4cb14fc8af61af2f6571a881 P 6faa80acac7144b7965315c270d55ccc: Bootstrap starting.
I20260811 09:07:38.787267 31266 tablet_bootstrap.cc:654] T f987040e4cb14fc8af61af2f6571a881 P 55d7238af4e440afa4c0803c769ddb9e: Neither blocks nor log segments found. Creating new log.
I20260811 09:07:38.789314 31269 tablet_bootstrap.cc:654] T f987040e4cb14fc8af61af2f6571a881 P 74fb90795ebb461aa84b8b6d1d9b2851: Neither blocks nor log segments found. Creating new log.
I20260811 09:07:38.789615 31268 tablet_bootstrap.cc:654] T f987040e4cb14fc8af61af2f6571a881 P 6faa80acac7144b7965315c270d55ccc: Neither blocks nor log segments found. Creating new log.
I20260811 09:07:38.789801 31266 log.cc:824] T f987040e4cb14fc8af61af2f6571a881 P 55d7238af4e440afa4c0803c769ddb9e: Log is configured to *not* fsync() on all Append() calls
I20260811 09:07:38.790746 31269 log.cc:824] T f987040e4cb14fc8af61af2f6571a881 P 74fb90795ebb461aa84b8b6d1d9b2851: Log is configured to *not* fsync() on all Append() calls
I20260811 09:07:38.790975 31268 log.cc:824] T f987040e4cb14fc8af61af2f6571a881 P 6faa80acac7144b7965315c270d55ccc: Log is configured to *not* fsync() on all Append() calls
I20260811 09:07:38.793403 31266 tablet_bootstrap.cc:492] T f987040e4cb14fc8af61af2f6571a881 P 55d7238af4e440afa4c0803c769ddb9e: No bootstrap required, opened a new log
I20260811 09:07:38.793704 31266 ts_tablet_manager.cc:1397] T f987040e4cb14fc8af61af2f6571a881 P 55d7238af4e440afa4c0803c769ddb9e: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260811 09:07:38.794157 31269 tablet_bootstrap.cc:492] T f987040e4cb14fc8af61af2f6571a881 P 74fb90795ebb461aa84b8b6d1d9b2851: No bootstrap required, opened a new log
I20260811 09:07:38.794171 31268 tablet_bootstrap.cc:492] T f987040e4cb14fc8af61af2f6571a881 P 6faa80acac7144b7965315c270d55ccc: No bootstrap required, opened a new log
I20260811 09:07:38.794435 31268 ts_tablet_manager.cc:1397] T f987040e4cb14fc8af61af2f6571a881 P 6faa80acac7144b7965315c270d55ccc: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260811 09:07:38.794433 31269 ts_tablet_manager.cc:1397] T f987040e4cb14fc8af61af2f6571a881 P 74fb90795ebb461aa84b8b6d1d9b2851: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.003s
I20260811 09:07:38.805101 31266 raft_consensus.cc:348] T f987040e4cb14fc8af61af2f6571a881 P 55d7238af4e440afa4c0803c769ddb9e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55d7238af4e440afa4c0803c769ddb9e" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 45879 } } peers { permanent_uuid: "74fb90795ebb461aa84b8b6d1d9b2851" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 44967 } } peers { permanent_uuid: "6faa80acac7144b7965315c270d55ccc" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 43949 } }
I20260811 09:07:38.805521 31266 raft_consensus.cc:374] T f987040e4cb14fc8af61af2f6571a881 P 55d7238af4e440afa4c0803c769ddb9e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:07:38.805646 31266 raft_consensus.cc:729] T f987040e4cb14fc8af61af2f6571a881 P 55d7238af4e440afa4c0803c769ddb9e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55d7238af4e440afa4c0803c769ddb9e, State: Initialized, Role: FOLLOWER
I20260811 09:07:38.805404 31268 raft_consensus.cc:348] T f987040e4cb14fc8af61af2f6571a881 P 6faa80acac7144b7965315c270d55ccc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55d7238af4e440afa4c0803c769ddb9e" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 45879 } } peers { permanent_uuid: "74fb90795ebb461aa84b8b6d1d9b2851" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 44967 } } peers { permanent_uuid: "6faa80acac7144b7965315c270d55ccc" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 43949 } }
I20260811 09:07:38.805883 31268 raft_consensus.cc:374] T f987040e4cb14fc8af61af2f6571a881 P 6faa80acac7144b7965315c270d55ccc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:07:38.805667 31269 raft_consensus.cc:348] T f987040e4cb14fc8af61af2f6571a881 P 74fb90795ebb461aa84b8b6d1d9b2851 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55d7238af4e440afa4c0803c769ddb9e" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 45879 } } peers { permanent_uuid: "74fb90795ebb461aa84b8b6d1d9b2851" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 44967 } } peers { permanent_uuid: "6faa80acac7144b7965315c270d55ccc" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 43949 } }
I20260811 09:07:38.806063 31268 raft_consensus.cc:729] T f987040e4cb14fc8af61af2f6571a881 P 6faa80acac7144b7965315c270d55ccc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6faa80acac7144b7965315c270d55ccc, State: Initialized, Role: FOLLOWER
I20260811 09:07:38.806133 31269 raft_consensus.cc:374] T f987040e4cb14fc8af61af2f6571a881 P 74fb90795ebb461aa84b8b6d1d9b2851 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:07:38.806252 31269 raft_consensus.cc:729] T f987040e4cb14fc8af61af2f6571a881 P 74fb90795ebb461aa84b8b6d1d9b2851 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74fb90795ebb461aa84b8b6d1d9b2851, State: Initialized, Role: FOLLOWER
I20260811 09:07:38.806411 31266 consensus_queue.cc:260] T f987040e4cb14fc8af61af2f6571a881 P 55d7238af4e440afa4c0803c769ddb9e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55d7238af4e440afa4c0803c769ddb9e" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 45879 } } peers { permanent_uuid: "74fb90795ebb461aa84b8b6d1d9b2851" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 44967 } } peers { permanent_uuid: "6faa80acac7144b7965315c270d55ccc" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 43949 } }
I20260811 09:07:38.806674 31268 consensus_queue.cc:260] T f987040e4cb14fc8af61af2f6571a881 P 6faa80acac7144b7965315c270d55ccc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55d7238af4e440afa4c0803c769ddb9e" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 45879 } } peers { permanent_uuid: "74fb90795ebb461aa84b8b6d1d9b2851" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 44967 } } peers { permanent_uuid: "6faa80acac7144b7965315c270d55ccc" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 43949 } }
I20260811 09:07:38.806903 31269 consensus_queue.cc:260] T f987040e4cb14fc8af61af2f6571a881 P 74fb90795ebb461aa84b8b6d1d9b2851 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55d7238af4e440afa4c0803c769ddb9e" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 45879 } } peers { permanent_uuid: "74fb90795ebb461aa84b8b6d1d9b2851" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 44967 } } peers { permanent_uuid: "6faa80acac7144b7965315c270d55ccc" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 43949 } }
I20260811 09:07:38.808508 31225 heartbeater.cc:503] Master 127.28.154.190:34395 was elected leader, sending a full tablet report...
I20260811 09:07:38.808935 31269 ts_tablet_manager.cc:1428] T f987040e4cb14fc8af61af2f6571a881 P 74fb90795ebb461aa84b8b6d1d9b2851: Time spent starting tablet: real 0.014s user 0.015s sys 0.001s
I20260811 09:07:38.810755 30972 heartbeater.cc:503] Master 127.28.154.190:34395 was elected leader, sending a full tablet report...
I20260811 09:07:38.811707 31268 ts_tablet_manager.cc:1428] T f987040e4cb14fc8af61af2f6571a881 P 6faa80acac7144b7965315c270d55ccc: Time spent starting tablet: real 0.017s user 0.014s sys 0.003s
I20260811 09:07:38.813490 31266 ts_tablet_manager.cc:1428] T f987040e4cb14fc8af61af2f6571a881 P 55d7238af4e440afa4c0803c769ddb9e: Time spent starting tablet: real 0.020s user 0.015s sys 0.004s
W20260811 09:07:38.826454 30723 tablet.cc:2369] T f987040e4cb14fc8af61af2f6571a881 P 55d7238af4e440afa4c0803c769ddb9e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 09:07:38.838722 30973 tablet.cc:2369] T f987040e4cb14fc8af61af2f6571a881 P 6faa80acac7144b7965315c270d55ccc: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 09:07:38.906409 31226 tablet.cc:2369] T f987040e4cb14fc8af61af2f6571a881 P 74fb90795ebb461aa84b8b6d1d9b2851: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 09:07:38.995662 31274 raft_consensus.cc:482] T f987040e4cb14fc8af61af2f6571a881 P 6faa80acac7144b7965315c270d55ccc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:07:38.996094 31274 raft_consensus.cc:504] T f987040e4cb14fc8af61af2f6571a881 P 6faa80acac7144b7965315c270d55ccc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55d7238af4e440afa4c0803c769ddb9e" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 45879 } } peers { permanent_uuid: "74fb90795ebb461aa84b8b6d1d9b2851" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 44967 } } peers { permanent_uuid: "6faa80acac7144b7965315c270d55ccc" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 43949 } }
I20260811 09:07:38.998631 31274 leader_election.cc:288] T f987040e4cb14fc8af61af2f6571a881 P 6faa80acac7144b7965315c270d55ccc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 55d7238af4e440afa4c0803c769ddb9e (127.28.154.129:45879), 74fb90795ebb461aa84b8b6d1d9b2851 (127.28.154.131:44967)
I20260811 09:07:39.005759 31258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:38.999039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.130:41327 (local address 127.28.154.129:45879)
0811 09:07:38.999230 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:38.999235 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:39.000053 (+ 818us) server_negotiation.cc:407] Connection header received
0811 09:07:39.000153 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:39.000159 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:39.000254 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:39.000396 (+ 142us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:07:39.001195 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:39.001958 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:39.003885 (+ 1927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:39.004848 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:39.005017 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:39.005099 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:39.005372 (+ 273us) server_negotiation.cc:299] Negotiation successful
0811 09:07:39.005505 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 09:07:39.005863 31276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:38.999045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.129:45879 (local address 127.28.154.130:41327)
0811 09:07:38.999798 (+ 753us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:38.999818 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 09:07:38.999986 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:07:39.000569 (+ 583us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:07:39.000573 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:07:39.000594 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:07:39.001046 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:39.001055 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:39.002088 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:39.002091 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:39.003710 (+ 1619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:39.003730 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:39.005071 (+ 1341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:39.005077 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:39.005218 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:39.005243 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 09:07:39.005333 (+ 90us) client_negotiation.cc:240] Negotiation successful
0811 09:07:39.005493 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":646,"thread_start_us":456,"threads_started":1}
I20260811 09:07:39.006047 31265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:39.000265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.130:46967 (local address 127.28.154.131:44967)
0811 09:07:39.000511 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:39.000517 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:39.000735 (+ 218us) server_negotiation.cc:407] Connection header received
0811 09:07:39.000946 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:39.000950 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:39.001010 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:39.001085 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:07:39.001658 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:39.002413 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:39.004196 (+ 1783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:39.005051 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:39.005207 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:39.005281 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:39.005606 (+ 325us) server_negotiation.cc:299] Negotiation successful
0811 09:07:39.005736 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260811 09:07:39.006247 31278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:38.999983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.131:44967 (local address 127.28.154.130:46967)
0811 09:07:39.000642 (+ 659us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:39.000656 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 09:07:39.000800 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:07:39.001114 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:07:39.001118 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:07:39.001140 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:07:39.001508 (+ 368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:39.001518 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:39.002557 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:39.002562 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:39.004038 (+ 1476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:39.004052 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:39.005275 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:39.005281 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:39.005456 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:39.005483 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 09:07:39.005564 (+ 81us) client_negotiation.cc:240] Negotiation successful
0811 09:07:39.005696 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":552,"thread_start_us":351,"threads_started":1}
I20260811 09:07:39.007164 30636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f987040e4cb14fc8af61af2f6571a881" candidate_uuid: "6faa80acac7144b7965315c270d55ccc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55d7238af4e440afa4c0803c769ddb9e" is_pre_election: true
I20260811 09:07:39.007302 31147 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f987040e4cb14fc8af61af2f6571a881" candidate_uuid: "6faa80acac7144b7965315c270d55ccc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74fb90795ebb461aa84b8b6d1d9b2851" is_pre_election: true
I20260811 09:07:39.007969 30636 raft_consensus.cc:2454] T f987040e4cb14fc8af61af2f6571a881 P 55d7238af4e440afa4c0803c769ddb9e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6faa80acac7144b7965315c270d55ccc in term 0.
I20260811 09:07:39.007967 31147 raft_consensus.cc:2454] T f987040e4cb14fc8af61af2f6571a881 P 74fb90795ebb461aa84b8b6d1d9b2851 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6faa80acac7144b7965315c270d55ccc in term 0.
I20260811 09:07:39.009074 30756 leader_election.cc:302] T f987040e4cb14fc8af61af2f6571a881 P 6faa80acac7144b7965315c270d55ccc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55d7238af4e440afa4c0803c769ddb9e, 6faa80acac7144b7965315c270d55ccc; no voters:
I20260811 09:07:39.009521 31274 raft_consensus.cc:2781] T f987040e4cb14fc8af61af2f6571a881 P 6faa80acac7144b7965315c270d55ccc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 09:07:39.009716 31274 raft_consensus.cc:482] T f987040e4cb14fc8af61af2f6571a881 P 6faa80acac7144b7965315c270d55ccc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 09:07:39.009838 31274 raft_consensus.cc:3037] T f987040e4cb14fc8af61af2f6571a881 P 6faa80acac7144b7965315c270d55ccc [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:07:39.012363 31274 raft_consensus.cc:504] T f987040e4cb14fc8af61af2f6571a881 P 6faa80acac7144b7965315c270d55ccc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55d7238af4e440afa4c0803c769ddb9e" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 45879 } } peers { permanent_uuid: "74fb90795ebb461aa84b8b6d1d9b2851" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 44967 } } peers { permanent_uuid: "6faa80acac7144b7965315c270d55ccc" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 43949 } }
I20260811 09:07:39.013193 31274 leader_election.cc:288] T f987040e4cb14fc8af61af2f6571a881 P 6faa80acac7144b7965315c270d55ccc [CANDIDATE]: Term 1 election: Requested vote from peers 55d7238af4e440afa4c0803c769ddb9e (127.28.154.129:45879), 74fb90795ebb461aa84b8b6d1d9b2851 (127.28.154.131:44967)
I20260811 09:07:39.013865 30636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f987040e4cb14fc8af61af2f6571a881" candidate_uuid: "6faa80acac7144b7965315c270d55ccc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55d7238af4e440afa4c0803c769ddb9e"
I20260811 09:07:39.013914 31147 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f987040e4cb14fc8af61af2f6571a881" candidate_uuid: "6faa80acac7144b7965315c270d55ccc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74fb90795ebb461aa84b8b6d1d9b2851"
I20260811 09:07:39.014127 30636 raft_consensus.cc:3037] T f987040e4cb14fc8af61af2f6571a881 P 55d7238af4e440afa4c0803c769ddb9e [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:07:39.014124 31147 raft_consensus.cc:3037] T f987040e4cb14fc8af61af2f6571a881 P 74fb90795ebb461aa84b8b6d1d9b2851 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:07:39.017786 31147 raft_consensus.cc:2454] T f987040e4cb14fc8af61af2f6571a881 P 74fb90795ebb461aa84b8b6d1d9b2851 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6faa80acac7144b7965315c270d55ccc in term 1.
I20260811 09:07:39.017894 30636 raft_consensus.cc:2454] T f987040e4cb14fc8af61af2f6571a881 P 55d7238af4e440afa4c0803c769ddb9e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6faa80acac7144b7965315c270d55ccc in term 1.
I20260811 09:07:39.018306 30756 leader_election.cc:302] T f987040e4cb14fc8af61af2f6571a881 P 6faa80acac7144b7965315c270d55ccc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6faa80acac7144b7965315c270d55ccc, 74fb90795ebb461aa84b8b6d1d9b2851; no voters:
I20260811 09:07:39.018627 31274 raft_consensus.cc:2781] T f987040e4cb14fc8af61af2f6571a881 P 6faa80acac7144b7965315c270d55ccc [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:07:39.019134 31274 raft_consensus.cc:686] T f987040e4cb14fc8af61af2f6571a881 P 6faa80acac7144b7965315c270d55ccc [term 1 LEADER]: Becoming Leader. State: Replica: 6faa80acac7144b7965315c270d55ccc, State: Running, Role: LEADER
I20260811 09:07:39.019707 31274 consensus_queue.cc:237] T f987040e4cb14fc8af61af2f6571a881 P 6faa80acac7144b7965315c270d55ccc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55d7238af4e440afa4c0803c769ddb9e" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 45879 } } peers { permanent_uuid: "74fb90795ebb461aa84b8b6d1d9b2851" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 44967 } } peers { permanent_uuid: "6faa80acac7144b7965315c270d55ccc" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 43949 } }
I20260811 09:07:39.025214 30347 catalog_manager.cc:5495] T f987040e4cb14fc8af61af2f6571a881 P 6faa80acac7144b7965315c270d55ccc reported cstate change: term changed from 0 to 1, leader changed from <none> to 6faa80acac7144b7965315c270d55ccc (127.28.154.130). New cstate: current_term: 1 leader_uuid: "6faa80acac7144b7965315c270d55ccc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55d7238af4e440afa4c0803c769ddb9e" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 45879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74fb90795ebb461aa84b8b6d1d9b2851" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 44967 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6faa80acac7144b7965315c270d55ccc" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 43949 } health_report { overall_health: HEALTHY } } }
Aug 11 09:07:39 dist-test-slave-xt4k krb5kdc[30260](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439256, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.154.129@KRBTEST.COM
I20260811 09:07:39.067631 31258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:39.051761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42990 (local address 127.28.154.129:45879)
0811 09:07:39.051997 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:39.052004 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:39.052547 (+ 543us) server_negotiation.cc:407] Connection header received
0811 09:07:39.052710 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:39.052714 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:39.052781 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:39.052871 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:39.054400 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:39.055271 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:39.056673 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:39.057106 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:39.057330 (+ 224us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:39.061452 (+ 4122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:07:39.061470 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:07:39.061488 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:07:39.061529 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:07:39.064847 (+ 3318us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:39.065489 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:39.065495 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:39.065498 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:39.065667 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:39.066123 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:39.066128 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:39.066130 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:39.066564 (+ 434us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:07:39.066812 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:39.067064 (+ 252us) server_negotiation.cc:299] Negotiation successful
0811 09:07:39.067239 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
Aug 11 09:07:39 dist-test-slave-xt4k krb5kdc[30260](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439259, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 09:07:39 dist-test-slave-xt4k krb5kdc[30260](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439259, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.28.154.129@KRBTEST.COM
I20260811 09:07:39.126511 31258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:39.109092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43006 (local address 127.28.154.129:45879)
0811 09:07:39.109327 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:39.109333 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:39.109593 (+ 260us) server_negotiation.cc:407] Connection header received
0811 09:07:39.109932 (+ 339us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:39.109939 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:39.110052 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:39.110195 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:39.112267 (+ 2072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:39.113352 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:39.115269 (+ 1917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:39.115769 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:39.115882 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:39.120840 (+ 4958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:07:39.120847 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:07:39.120851 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:07:39.120897 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:07:39.123718 (+ 2821us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:39.124518 (+ 800us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:39.124523 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:39.124525 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:39.124686 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:39.125141 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:39.125144 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:39.125146 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:39.125440 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:07:39.125763 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:39.125982 (+ 219us) server_negotiation.cc:299] Negotiation successful
0811 09:07:39.126182 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
Aug 11 09:07:39 dist-test-slave-xt4k krb5kdc[30260](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439259, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 09:07:39 dist-test-slave-xt4k krb5kdc[30260](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439259, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.154.129@KRBTEST.COM
I20260811 09:07:39.180856 31258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:39.165559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43008 (local address 127.28.154.129:45879)
0811 09:07:39.165775 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:39.165782 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:39.166128 (+ 346us) server_negotiation.cc:407] Connection header received
0811 09:07:39.166315 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:39.166320 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:39.166393 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:39.166493 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:39.168056 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:39.168891 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:39.170277 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:39.170739 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:39.170859 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:39.175470 (+ 4611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:07:39.175477 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:07:39.175480 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:07:39.175548 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:07:39.178186 (+ 2638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:39.178910 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:39.178916 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:39.178918 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:39.179059 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:39.179420 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:39.179425 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:39.179427 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:39.179761 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:07:39.179974 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:39.180233 (+ 259us) server_negotiation.cc:299] Negotiation successful
0811 09:07:39.180464 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
W20260811 09:07:39.183250 30545 tablet_service.cc:2908] Rejecting scan request for tablet f987040e4cb14fc8af61af2f6571a881: Uninitialized: clean time has not yet been initialized
I20260811 09:07:39.188162 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 30484
I20260811 09:07:39.202873 31258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:39.189096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43024 (local address 127.28.154.129:45879)
0811 09:07:39.189299 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:39.189306 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:39.189769 (+ 463us) server_negotiation.cc:407] Connection header received
0811 09:07:39.189845 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:39.189849 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:39.189943 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:39.190031 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:39.191469 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:39.192224 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:39.193499 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:39.194137 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:39.194343 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:39.196828 (+ 2485us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:07:39.196834 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:07:39.196838 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:07:39.196875 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:07:39.200118 (+ 3243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:39.200864 (+ 746us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:39.200869 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:39.200871 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:39.201038 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:39.201387 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:39.201390 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:39.201392 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:39.201798 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:07:39.202045 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:39.202418 (+ 373us) server_negotiation.cc:299] Negotiation successful
0811 09:07:39.202613 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 09:07:39.203522 30711 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:07:39.292593 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 30484
I20260811 09:07:39.314617 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 30731
Aug 11 09:07:39 dist-test-slave-xt4k krb5kdc[30260](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439259, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.154.130@KRBTEST.COM
I20260811 09:07:39.330317 31308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:39.315460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41000 (local address 127.28.154.130:43949)
0811 09:07:39.315907 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:39.315911 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:39.315926 (+ 15us) server_negotiation.cc:407] Connection header received
0811 09:07:39.316001 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:39.316005 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:39.316066 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:39.316175 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:39.317450 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:39.318158 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:39.319434 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:39.319825 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:39.319938 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:39.324283 (+ 4345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:07:39.324318 (+ 35us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:07:39.324352 (+ 34us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:07:39.324395 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:07:39.327326 (+ 2931us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:39.328108 (+ 782us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:39.328116 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:39.328119 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:39.328281 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:39.328818 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:39.328824 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:39.328826 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:39.329273 (+ 447us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:07:39.329535 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:39.329798 (+ 263us) server_negotiation.cc:299] Negotiation successful
0811 09:07:39.330004 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349,"thread_start_us":219,"threads_started":1}
I20260811 09:07:39.331032 30967 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:07:39.421468 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 30731
I20260811 09:07:39.445978 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 30978
Aug 11 09:07:39 dist-test-slave-xt4k krb5kdc[30260](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439259, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.154.131@KRBTEST.COM
I20260811 09:07:39.462208 31265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:39.446831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52418 (local address 127.28.154.131:44967)
0811 09:07:39.447054 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:39.447060 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:39.447082 (+ 22us) server_negotiation.cc:407] Connection header received
0811 09:07:39.447211 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:39.447217 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:39.447287 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:39.447374 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:39.448745 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:39.449587 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:39.450933 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:39.451324 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:39.451436 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:39.456242 (+ 4806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:07:39.456260 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:07:39.456279 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:07:39.456327 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:07:39.459193 (+ 2866us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:39.459928 (+ 735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:39.459936 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:39.459938 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:39.460084 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:39.460621 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:39.460626 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:39.460628 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:39.461012 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:07:39.461288 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:39.461634 (+ 346us) server_negotiation.cc:299] Negotiation successful
0811 09:07:39.461859 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 09:07:39.462920 31217 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:07:39.551549 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 30978
I20260811 09:07:39.575027 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 30286
I20260811 09:07:39.576259 30421 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:07:39.685231 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 30286
2026-08-11T09:07:39Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (4649 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439250192320-29290-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:07:39 dist-test-slave-xt4k krb5kdc[31315](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:07:39 dist-test-slave-xt4k krb5kdc[31315](info): set up 2 sockets
Aug 11 09:07:39 dist-test-slave-xt4k krb5kdc[31315](info): commencing operation
krb5kdc: starting...
W20260811 09:07:40.968176 29290 mini_kdc.cc:121] Time spent starting KDC: real 1.231s user 0.002s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439250192320-29290-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439250192320-29290-0/krb5kdc/test-user.keytab.
Aug 11 09:07:41 dist-test-slave-xt4k krb5kdc[31315](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439261, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:07:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:07:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.154.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.154.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.154.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab.
Entry for principal kudu/127.28.154.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab.
I20260811 09:07:41.181314 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439250192320-29290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439250192320-29290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439250192320-29290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.154.190:45807
--webserver_interface=127.28.154.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab
--principal=kudu/127.28.154.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:33931
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.154.190:45807
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:07:41.516512 31340 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:07:41.516860 31340 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:07:41.516997 31340 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:07:41.524484 31340 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:07:41.524628 31340 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 09:07:41.524704 31340 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:07:41.524789 31340 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:07:41.524863 31340 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:07:41.524950 31340 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:07:41.535833 31340 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:33931
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439250192320-29290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439250192320-29290-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.154.190:45807
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab
--principal=kudu/127.28.154.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.154.190:45807
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.154.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439250192320-29290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:07:41.537204 31340 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:07:41.538568 31340 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:07:41.550370 31352 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:07:41.550323 31354 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:07:41.550522 31359 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:07:41.551551 31340 server_base.cc:1034] running on GCE node
I20260811 09:07:41.552383 31340 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:07:41.553858 31340 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:07:41.555099 31340 hybrid_clock.cc:648] HybridClock initialized: now 1786439261555020 us; error 41 us; skew 500 ppm
I20260811 09:07:41.555430 31340 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:07:41 dist-test-slave-xt4k krb5kdc[31315](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439261, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:07:41.558609 31340 init.cc:377] Logged in from keytab as kudu/127.28.154.190@KRBTEST.COM (short username kudu)
I20260811 09:07:41.560182 31340 webserver.cc:460] Webserver started at http://127.28.154.190:39937/ using document root <none> and password file <none>
I20260811 09:07:41.560959 31340 fs_manager.cc:359] Metadata directory not provided
I20260811 09:07:41.561065 31340 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:07:41.561381 31340 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:07:41.564420 31340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439250192320-29290-0/minicluster-data/master-0/data/instance:
uuid: "f6cfaebcd30e476f93f1c0f1154871c7"
format_stamp: "Formatted at 2026-08-11 09:07:41 on dist-test-slave-xt4k"
server_key: "1139ecd906a870bcf6b97682c22acc9d"
server_key_iv: "4a1b541021d36f241db4ef30557d8329"
server_key_version: "encryptionkey@0"
I20260811 09:07:41.565203 31340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439250192320-29290-0/minicluster-data/master-0/wal/instance:
uuid: "f6cfaebcd30e476f93f1c0f1154871c7"
format_stamp: "Formatted at 2026-08-11 09:07:41 on dist-test-slave-xt4k"
server_key: "1139ecd906a870bcf6b97682c22acc9d"
server_key_iv: "4a1b541021d36f241db4ef30557d8329"
server_key_version: "encryptionkey@0"
I20260811 09:07:41.570876 31340 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 09:07:41.574534 31377 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:07:41.575912 31340 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260811 09:07:41.576095 31340 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439250192320-29290-0/minicluster-data/master-0/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439250192320-29290-0/minicluster-data/master-0/wal
uuid: "f6cfaebcd30e476f93f1c0f1154871c7"
format_stamp: "Formatted at 2026-08-11 09:07:41 on dist-test-slave-xt4k"
server_key: "1139ecd906a870bcf6b97682c22acc9d"
server_key_iv: "4a1b541021d36f241db4ef30557d8329"
server_key_version: "encryptionkey@0"
I20260811 09:07:41.576309 31340 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439250192320-29290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439250192320-29290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439250192320-29290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:07:41.598647 31340 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:07:41.603158 31340 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:07:41.603427 31340 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:07:41.624608 31340 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.190:45807
I20260811 09:07:41.624629 31473 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.190:45807 every 8 connection(s)
I20260811 09:07:41.626194 31340 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
I20260811 09:07:41.630066 31474 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:07:41.631940 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 31340
I20260811 09:07:41.632172 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439250192320-29290-0/minicluster-data/master-0/wal/instance
I20260811 09:07:41.633033 29290 external_mini_cluster.cc:1428] Setting key 3b13c6f32c825a96dc935ca8e800e6b7
I20260811 09:07:41.642979 31474 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6cfaebcd30e476f93f1c0f1154871c7: Bootstrap starting.
I20260811 09:07:41.646965 31474 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f6cfaebcd30e476f93f1c0f1154871c7: Neither blocks nor log segments found. Creating new log.
Aug 11 09:07:41 dist-test-slave-xt4k krb5kdc[31315](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439261, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.154.190@KRBTEST.COM
I20260811 09:07:41.648136 31474 log.cc:824] T 00000000000000000000000000000000 P f6cfaebcd30e476f93f1c0f1154871c7: Log is configured to *not* fsync() on all Append() calls
I20260811 09:07:41.651422 31474 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6cfaebcd30e476f93f1c0f1154871c7: No bootstrap required, opened a new log
I20260811 09:07:41.656673 31483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:41.635146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38216 (local address 127.28.154.190:45807)
0811 09:07:41.636149 (+ 1003us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:41.636161 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:41.636211 (+ 50us) server_negotiation.cc:407] Connection header received
0811 09:07:41.637936 (+ 1725us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:41.637969 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:41.638358 (+ 389us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:41.638695 (+ 337us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:41.640439 (+ 1744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:41.641783 (+ 1344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:41.643479 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:41.644190 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:41.644329 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:41.649244 (+ 4915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:07:41.649260 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:07:41.649281 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:07:41.649333 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:07:41.652518 (+ 3185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:41.653385 (+ 867us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:41.653392 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:41.653399 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:41.653566 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:41.653976 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:41.653980 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:41.653982 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:41.654539 (+ 557us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:07:41.654767 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:41.655217 (+ 450us) server_negotiation.cc:299] Negotiation successful
0811 09:07:41.655571 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":791,"thread_start_us":501,"threads_started":1}
I20260811 09:07:41.659749 31474 raft_consensus.cc:348] T 00000000000000000000000000000000 P f6cfaebcd30e476f93f1c0f1154871c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6cfaebcd30e476f93f1c0f1154871c7" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 45807 } }
I20260811 09:07:41.660096 31474 raft_consensus.cc:374] T 00000000000000000000000000000000 P f6cfaebcd30e476f93f1c0f1154871c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:07:41.660184 31474 raft_consensus.cc:729] T 00000000000000000000000000000000 P f6cfaebcd30e476f93f1c0f1154871c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6cfaebcd30e476f93f1c0f1154871c7, State: Initialized, Role: FOLLOWER
I20260811 09:07:41.660791 31474 consensus_queue.cc:260] T 00000000000000000000000000000000 P f6cfaebcd30e476f93f1c0f1154871c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6cfaebcd30e476f93f1c0f1154871c7" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 45807 } }
I20260811 09:07:41.661015 31474 raft_consensus.cc:388] T 00000000000000000000000000000000 P f6cfaebcd30e476f93f1c0f1154871c7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:07:41.661144 31474 raft_consensus.cc:482] T 00000000000000000000000000000000 P f6cfaebcd30e476f93f1c0f1154871c7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:07:41.661298 31474 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f6cfaebcd30e476f93f1c0f1154871c7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:07:41.663478 31474 raft_consensus.cc:504] T 00000000000000000000000000000000 P f6cfaebcd30e476f93f1c0f1154871c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6cfaebcd30e476f93f1c0f1154871c7" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 45807 } }
I20260811 09:07:41.663980 31474 leader_election.cc:302] T 00000000000000000000000000000000 P f6cfaebcd30e476f93f1c0f1154871c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f6cfaebcd30e476f93f1c0f1154871c7; no voters:
I20260811 09:07:41.664572 31474 leader_election.cc:288] T 00000000000000000000000000000000 P f6cfaebcd30e476f93f1c0f1154871c7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:07:41.664692 31488 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f6cfaebcd30e476f93f1c0f1154871c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:07:41.665410 31488 raft_consensus.cc:686] T 00000000000000000000000000000000 P f6cfaebcd30e476f93f1c0f1154871c7 [term 1 LEADER]: Becoming Leader. State: Replica: f6cfaebcd30e476f93f1c0f1154871c7, State: Running, Role: LEADER
I20260811 09:07:41.665954 31488 consensus_queue.cc:237] T 00000000000000000000000000000000 P f6cfaebcd30e476f93f1c0f1154871c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6cfaebcd30e476f93f1c0f1154871c7" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 45807 } }
I20260811 09:07:41.666591 31474 sys_catalog.cc:564] T 00000000000000000000000000000000 P f6cfaebcd30e476f93f1c0f1154871c7 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:07:41.669070 31491 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6cfaebcd30e476f93f1c0f1154871c7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f6cfaebcd30e476f93f1c0f1154871c7. Latest consensus state: current_term: 1 leader_uuid: "f6cfaebcd30e476f93f1c0f1154871c7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6cfaebcd30e476f93f1c0f1154871c7" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 45807 } } }
I20260811 09:07:41.669094 31490 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6cfaebcd30e476f93f1c0f1154871c7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f6cfaebcd30e476f93f1c0f1154871c7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f6cfaebcd30e476f93f1c0f1154871c7" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 45807 } } }
I20260811 09:07:41.669554 31491 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6cfaebcd30e476f93f1c0f1154871c7 [sys.catalog]: This master's current role is: LEADER
I20260811 09:07:41.669744 31490 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6cfaebcd30e476f93f1c0f1154871c7 [sys.catalog]: This master's current role is: LEADER
I20260811 09:07:41.678151 31497 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:07:41.683228 31497 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:07:41.691617 31497 catalog_manager.cc:1347] Generated new cluster ID: ce2ee7da2994422daefb9ee10ca14324
I20260811 09:07:41.691738 31497 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:07:41.704242 31497 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:07:41.705586 31497 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:07:41.715199 31497 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f6cfaebcd30e476f93f1c0f1154871c7: Generated new TSK 0
I20260811 09:07:41.716038 31497 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.28.154.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.154.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.154.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.129.keytab.
Entry for principal kudu/127.28.154.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.129.keytab.
I20260811 09:07:41.776834 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439250192320-29290-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439250192320-29290-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439250192320-29290-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439250192320-29290-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.154.129:0
--local_ip_for_outbound_sockets=127.28.154.129
--webserver_interface=127.28.154.129
--webserver_port=0
--tserver_master_addrs=127.28.154.190:45807
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.129.keytab
--principal=kudu/127.28.154.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:33931
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:07:42.115509 31517 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:07:42.115790 31517 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:07:42.115911 31517 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:07:42.123253 31517 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:07:42.123377 31517 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:07:42.123486 31517 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 09:07:42.123595 31517 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.154.129
I20260811 09:07:42.134114 31517 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:33931
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439250192320-29290-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439250192320-29290-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.129.keytab
--principal=kudu/127.28.154.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.154.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439250192320-29290-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.154.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.28.154.190:45807
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.154.129
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439250192320-29290-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:07:42.135480 31517 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:07:42.136983 31517 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:07:42.150269 31528 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:07:42.150290 31530 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:07:42.150228 31533 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:07:42.151363 31517 server_base.cc:1034] running on GCE node
I20260811 09:07:42.152009 31517 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:07:42.153018 31517 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:07:42.154245 31517 hybrid_clock.cc:648] HybridClock initialized: now 1786439262154170 us; error 48 us; skew 500 ppm
I20260811 09:07:42.154543 31517 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:07:42 dist-test-slave-xt4k krb5kdc[31315](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439262, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:07:42.164091 31517 init.cc:377] Logged in from keytab as kudu/127.28.154.129@KRBTEST.COM (short username kudu)
I20260811 09:07:42.165832 31517 webserver.cc:460] Webserver started at http://127.28.154.129:39973/ using document root <none> and password file <none>
I20260811 09:07:42.166594 31517 fs_manager.cc:359] Metadata directory not provided
I20260811 09:07:42.166685 31517 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:07:42.167025 31517 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:07:42.170138 31517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439250192320-29290-0/minicluster-data/ts-0/data/instance:
uuid: "44fb00a4d0c3453bb1a53ef91dc21027"
format_stamp: "Formatted at 2026-08-11 09:07:42 on dist-test-slave-xt4k"
server_key: "a848d21a9e938dc52aebcede3fa12f54"
server_key_iv: "3907578b44a5591a6b567910909afa96"
server_key_version: "encryptionkey@0"
I20260811 09:07:42.170972 31517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439250192320-29290-0/minicluster-data/ts-0/wal/instance:
uuid: "44fb00a4d0c3453bb1a53ef91dc21027"
format_stamp: "Formatted at 2026-08-11 09:07:42 on dist-test-slave-xt4k"
server_key: "a848d21a9e938dc52aebcede3fa12f54"
server_key_iv: "3907578b44a5591a6b567910909afa96"
server_key_version: "encryptionkey@0"
I20260811 09:07:42.176748 31517 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260811 09:07:42.180754 31542 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:07:42.182232 31517 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 09:07:42.182438 31517 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439250192320-29290-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439250192320-29290-0/minicluster-data/ts-0/wal
uuid: "44fb00a4d0c3453bb1a53ef91dc21027"
format_stamp: "Formatted at 2026-08-11 09:07:42 on dist-test-slave-xt4k"
server_key: "a848d21a9e938dc52aebcede3fa12f54"
server_key_iv: "3907578b44a5591a6b567910909afa96"
server_key_version: "encryptionkey@0"
I20260811 09:07:42.182670 31517 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439250192320-29290-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439250192320-29290-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439250192320-29290-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:07:42.228459 31517 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:07:42.234251 31517 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:07:42.234527 31517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:07:42.237524 31517 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:07:42.237617 31517 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:07:42.237740 31517 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:07:42.237793 31517 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 09:07:42 dist-test-slave-xt4k krb5kdc[31315](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439262, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.129@KRBTEST.COM for kudu/127.28.154.190@KRBTEST.COM
I20260811 09:07:42.262401 31561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:42.239227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.129:41845 (local address 127.28.154.190:45807)
0811 09:07:42.244708 (+ 5481us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:42.244713 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:42.244735 (+ 22us) server_negotiation.cc:407] Connection header received
0811 09:07:42.244791 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:42.244796 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:42.244863 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:42.244973 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:42.246722 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:42.247492 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:42.249646 (+ 2154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:42.250211 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:42.250348 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:42.255415 (+ 5067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:07:42.255422 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:07:42.255427 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:07:42.255474 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:07:42.258422 (+ 2948us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:42.259584 (+ 1162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:42.259589 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:42.259591 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:42.259719 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:42.260266 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:42.260272 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:42.260275 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:42.260714 (+ 439us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:07:42.260874 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:42.261690 (+ 816us) server_negotiation.cc:299] Negotiation successful
0811 09:07:42.261967 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5359,"thread_start_us":5269,"threads_started":1}
I20260811 09:07:42.263165 31559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:42.238657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:45807 (local address 127.28.154.129:41845)
0811 09:07:42.239512 (+ 855us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:42.239582 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0811 09:07:42.241560 (+ 1978us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:07:42.245263 (+ 3703us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:07:42.245273 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:07:42.245870 (+ 597us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:07:42.246520 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:42.246549 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:42.247659 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:42.247663 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:42.249393 (+ 1730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:42.249415 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:42.250537 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:42.250547 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:42.250876 (+ 329us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:42.251862 (+ 986us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:07:42.251901 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:07:42.255132 (+ 3231us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:07:42.258580 (+ 3448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:07:42.258596 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:07:42.258622 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:07:42.259397 (+ 775us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:07:42.259836 (+ 439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:07:42.259841 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:07:42.259845 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:07:42.260090 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:07:42.260891 (+ 801us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:07:42.260900 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:07:42.261294 (+ 394us) client_negotiation.cc:769] Sending connection context
0811 09:07:42.261534 (+ 240us) client_negotiation.cc:240] Negotiation successful
0811 09:07:42.261963 (+ 429us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":555,"thread_start_us":342,"threads_started":1}
I20260811 09:07:42.284675 31561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:42.269347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.129:38257 (local address 127.28.154.190:45807)
0811 09:07:42.269527 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:42.269531 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:42.270107 (+ 576us) server_negotiation.cc:407] Connection header received
0811 09:07:42.270265 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:42.270271 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:42.270368 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:42.270481 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:42.271392 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:42.272589 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:42.274746 (+ 2157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:42.275487 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:42.275609 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:42.278351 (+ 2742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:07:42.278358 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:07:42.278363 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:07:42.278407 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:07:42.281685 (+ 3278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:42.282605 (+ 920us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:42.282611 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:42.282616 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:42.282758 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:42.283128 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:42.283134 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:42.283138 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:42.283425 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:07:42.283598 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:42.284042 (+ 444us) server_negotiation.cc:299] Negotiation successful
0811 09:07:42.284275 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 09:07:42.284821 31691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:42.269420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:45807 (local address 127.28.154.129:38257)
0811 09:07:42.270007 (+ 587us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:42.270026 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 09:07:42.270145 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:07:42.270488 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:07:42.270491 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:07:42.270955 (+ 464us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:07:42.271254 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:42.271264 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:42.272784 (+ 1520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:42.272792 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:42.274519 (+ 1727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:42.274540 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:42.275713 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:42.275720 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:42.275939 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:42.276754 (+ 815us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:07:42.276784 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:07:42.278054 (+ 1270us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:07:42.281819 (+ 3765us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:07:42.281826 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:07:42.281830 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:07:42.282469 (+ 639us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:07:42.282870 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:07:42.282875 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:07:42.282880 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:07:42.283020 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:07:42.283553 (+ 533us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:07:42.283558 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:07:42.283765 (+ 207us) client_negotiation.cc:769] Sending connection context
0811 09:07:42.283913 (+ 148us) client_negotiation.cc:240] Negotiation successful
0811 09:07:42.284276 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":317,"threads_started":1}
I20260811 09:07:42.297773 31517 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.129:34213
I20260811 09:07:42.297791 31818 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.129:34213 every 8 connection(s)
I20260811 09:07:42.299050 31517 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439250192320-29290-0/minicluster-data/ts-0/data/info.pb
I20260811 09:07:42.301184 31819 heartbeater.cc:347] Connected to a master server at 127.28.154.190:45807
I20260811 09:07:42.301403 31819 heartbeater.cc:464] Registering TS with master...
I20260811 09:07:42.302034 31819 heartbeater.cc:511] Master 127.28.154.190:45807 requested a full tablet report, sending...
I20260811 09:07:42.304098 31403 ts_manager.cc:194] Registered new tserver with Master: 44fb00a4d0c3453bb1a53ef91dc21027 (127.28.154.129:34213)
I20260811 09:07:42.305608 31403 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.154.129@KRBTEST.COM'} at 127.28.154.129:41845
I20260811 09:07:42.308642 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 31517
I20260811 09:07:42.308818 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439250192320-29290-0/minicluster-data/ts-0/wal/instance
I20260811 09:07:42.309422 29290 external_mini_cluster.cc:1428] Setting key 8262f830b4b9a7ef00c1e4f4158b057e
WARNING: no policy specified for kudu/127.28.154.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.154.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.154.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.130.keytab.
Entry for principal kudu/127.28.154.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.130.keytab.
I20260811 09:07:42.351624 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439250192320-29290-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439250192320-29290-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439250192320-29290-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439250192320-29290-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.154.130:0
--local_ip_for_outbound_sockets=127.28.154.130
--webserver_interface=127.28.154.130
--webserver_port=0
--tserver_master_addrs=127.28.154.190:45807
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.130.keytab
--principal=kudu/127.28.154.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:33931
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:07:42.677237 31825 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:07:42.677551 31825 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:07:42.677675 31825 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:07:42.685523 31825 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:07:42.685652 31825 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:07:42.685793 31825 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 09:07:42.685905 31825 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.154.130
I20260811 09:07:42.696808 31825 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:33931
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439250192320-29290-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439250192320-29290-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.130.keytab
--principal=kudu/127.28.154.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.154.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439250192320-29290-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.154.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.28.154.190:45807
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.154.130
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439250192320-29290-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:07:42.698194 31825 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:07:42.699573 31825 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:07:42.713871 31834 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:07:42.713912 31841 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:07:42.713935 31836 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:07:42.715108 31825 server_base.cc:1034] running on GCE node
I20260811 09:07:42.715910 31825 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:07:42.716868 31825 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:07:42.718130 31825 hybrid_clock.cc:648] HybridClock initialized: now 1786439262718039 us; error 42 us; skew 500 ppm
I20260811 09:07:42.718504 31825 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:07:42 dist-test-slave-xt4k krb5kdc[31315](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439262, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:07:42.721940 31825 init.cc:377] Logged in from keytab as kudu/127.28.154.130@KRBTEST.COM (short username kudu)
I20260811 09:07:42.723532 31825 webserver.cc:460] Webserver started at http://127.28.154.130:33395/ using document root <none> and password file <none>
I20260811 09:07:42.724409 31825 fs_manager.cc:359] Metadata directory not provided
I20260811 09:07:42.724535 31825 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:07:42.724874 31825 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:07:42.728017 31825 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439250192320-29290-0/minicluster-data/ts-1/data/instance:
uuid: "096a81e294374bb9aec1df09659f4d06"
format_stamp: "Formatted at 2026-08-11 09:07:42 on dist-test-slave-xt4k"
server_key: "4337da1f69aff5173c917eb521bced86"
server_key_iv: "1310e8bb4b8699545a5aad8562a20b32"
server_key_version: "encryptionkey@0"
I20260811 09:07:42.728951 31825 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439250192320-29290-0/minicluster-data/ts-1/wal/instance:
uuid: "096a81e294374bb9aec1df09659f4d06"
format_stamp: "Formatted at 2026-08-11 09:07:42 on dist-test-slave-xt4k"
server_key: "4337da1f69aff5173c917eb521bced86"
server_key_iv: "1310e8bb4b8699545a5aad8562a20b32"
server_key_version: "encryptionkey@0"
I20260811 09:07:42.734496 31825 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 09:07:42.738474 31852 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:07:42.739944 31825 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 09:07:42.740130 31825 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439250192320-29290-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439250192320-29290-0/minicluster-data/ts-1/wal
uuid: "096a81e294374bb9aec1df09659f4d06"
format_stamp: "Formatted at 2026-08-11 09:07:42 on dist-test-slave-xt4k"
server_key: "4337da1f69aff5173c917eb521bced86"
server_key_iv: "1310e8bb4b8699545a5aad8562a20b32"
server_key_version: "encryptionkey@0"
I20260811 09:07:42.740383 31825 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439250192320-29290-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439250192320-29290-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439250192320-29290-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:07:42.756907 31825 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:07:42.761196 31825 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:07:42.761426 31825 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:07:42.764487 31825 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:07:42.764578 31825 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:07:42.764767 31825 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:07:42.764868 31825 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 09:07:42 dist-test-slave-xt4k krb5kdc[31315](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439262, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.130@KRBTEST.COM for kudu/127.28.154.190@KRBTEST.COM
I20260811 09:07:42.789192 31561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:42.764951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.130:40253 (local address 127.28.154.190:45807)
0811 09:07:42.765147 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:42.765154 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:42.766688 (+ 1534us) server_negotiation.cc:407] Connection header received
0811 09:07:42.768904 (+ 2216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:42.768912 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:42.769024 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:42.769153 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:42.771222 (+ 2069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:42.772203 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:42.774829 (+ 2626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:42.775432 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:42.775575 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:42.781473 (+ 5898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:07:42.781481 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:07:42.781486 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:07:42.781524 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:07:42.785261 (+ 3737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:42.786297 (+ 1036us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:42.786304 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:42.786307 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:42.786481 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:42.787031 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:42.787037 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:42.787041 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:42.787373 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:07:42.787554 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:42.788733 (+ 1179us) server_negotiation.cc:299] Negotiation successful
0811 09:07:42.788919 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 09:07:42.790910 31865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:42.765518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:45807 (local address 127.28.154.130:40253)
0811 09:07:42.766490 (+ 972us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:42.766535 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 09:07:42.768624 (+ 2089us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:07:42.769401 (+ 777us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:07:42.769414 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:07:42.770093 (+ 679us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:07:42.771026 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:42.771057 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:42.772423 (+ 1366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:42.772429 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:42.774679 (+ 2250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:42.774699 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:42.775713 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:42.775721 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:42.776073 (+ 352us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:42.776992 (+ 919us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:07:42.777032 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:07:42.781138 (+ 4106us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:07:42.785428 (+ 4290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:07:42.785439 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:07:42.785459 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:07:42.786091 (+ 632us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:07:42.786608 (+ 517us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:07:42.786613 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:07:42.786617 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:07:42.786909 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:07:42.787560 (+ 651us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:07:42.787571 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:07:42.788150 (+ 579us) client_negotiation.cc:769] Sending connection context
0811 09:07:42.788607 (+ 457us) client_negotiation.cc:240] Negotiation successful
0811 09:07:42.789277 (+ 670us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":607,"thread_start_us":389,"threads_started":1}
I20260811 09:07:42.818332 31998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:42.801405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:45807 (local address 127.28.154.130:49837)
0811 09:07:42.803574 (+ 2169us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:42.803598 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 09:07:42.803784 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:07:42.804409 (+ 625us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:07:42.804414 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:07:42.805046 (+ 632us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:07:42.805478 (+ 432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:42.805495 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:42.806585 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:42.806591 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:42.807680 (+ 1089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:42.807694 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:42.808577 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:42.808589 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:42.808806 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:42.809603 (+ 797us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:07:42.809631 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:07:42.810758 (+ 1127us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:07:42.815090 (+ 4332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:07:42.815096 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:07:42.815100 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:07:42.815771 (+ 671us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:07:42.816254 (+ 483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:07:42.816259 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:07:42.816266 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:07:42.816497 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:07:42.817235 (+ 738us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:07:42.817240 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:07:42.817487 (+ 247us) client_negotiation.cc:769] Sending connection context
0811 09:07:42.817688 (+ 201us) client_negotiation.cc:240] Negotiation successful
0811 09:07:42.817977 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1954,"thread_start_us":1772,"threads_started":1}
I20260811 09:07:42.818485 31561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:42.800225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.130:49837 (local address 127.28.154.190:45807)
0811 09:07:42.800447 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:42.800453 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:42.803725 (+ 3272us) server_negotiation.cc:407] Connection header received
0811 09:07:42.803965 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:42.803971 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:42.804077 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:42.804192 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:42.805652 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:42.806427 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:42.807820 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:42.808291 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:42.808465 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:42.810996 (+ 2531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:07:42.811001 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:07:42.811005 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:07:42.811044 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:07:42.814911 (+ 3867us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:42.815931 (+ 1020us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:42.815935 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:42.815938 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:42.816114 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:42.816632 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:42.816637 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:42.816640 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:42.817118 (+ 478us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:07:42.817257 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:42.817852 (+ 595us) server_negotiation.cc:299] Negotiation successful
0811 09:07:42.818111 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 09:07:42.825641 31825 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.130:34727
I20260811 09:07:42.825760 32065 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.130:34727 every 8 connection(s)
I20260811 09:07:42.826875 31825 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439250192320-29290-0/minicluster-data/ts-1/data/info.pb
I20260811 09:07:42.828981 32066 heartbeater.cc:347] Connected to a master server at 127.28.154.190:45807
I20260811 09:07:42.829196 32066 heartbeater.cc:464] Registering TS with master...
I20260811 09:07:42.829813 32066 heartbeater.cc:511] Master 127.28.154.190:45807 requested a full tablet report, sending...
I20260811 09:07:42.831038 31403 ts_manager.cc:194] Registered new tserver with Master: 096a81e294374bb9aec1df09659f4d06 (127.28.154.130:34727)
I20260811 09:07:42.831862 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 31825
I20260811 09:07:42.832053 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439250192320-29290-0/minicluster-data/ts-1/wal/instance
I20260811 09:07:42.832058 31403 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.154.130@KRBTEST.COM'} at 127.28.154.130:40253
I20260811 09:07:42.832628 29290 external_mini_cluster.cc:1428] Setting key 691df0354385df3d16bb549f0b96c7ac
WARNING: no policy specified for kudu/127.28.154.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.154.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.154.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.131.keytab.
Entry for principal kudu/127.28.154.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.131.keytab.
I20260811 09:07:42.875644 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439250192320-29290-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439250192320-29290-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439250192320-29290-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439250192320-29290-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.154.131:0
--local_ip_for_outbound_sockets=127.28.154.131
--webserver_interface=127.28.154.131
--webserver_port=0
--tserver_master_addrs=127.28.154.190:45807
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.131.keytab
--principal=kudu/127.28.154.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:33931
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:07:43.253906 32072 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:07:43.254209 32072 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:07:43.254338 32072 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:07:43.262276 32072 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:07:43.262419 32072 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:07:43.262557 32072 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 09:07:43.262673 32072 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.154.131
I20260811 09:07:43.273288 32072 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:33931
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439250192320-29290-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439250192320-29290-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.131.keytab
--principal=kudu/127.28.154.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.154.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439250192320-29290-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.154.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.28.154.190:45807
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.154.131
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439250192320-29290-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:07:43.274613 32072 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:07:43.276237 32072 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:07:43.291167 32083 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:07:43.291137 32085 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:07:43.291100 32091 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:07:43.292711 32072 server_base.cc:1034] running on GCE node
I20260811 09:07:43.293424 32072 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:07:43.294519 32072 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:07:43.295849 32072 hybrid_clock.cc:648] HybridClock initialized: now 1786439263295769 us; error 37 us; skew 500 ppm
I20260811 09:07:43.296221 32072 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:07:43 dist-test-slave-xt4k krb5kdc[31315](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439263, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:07:43.300948 32072 init.cc:377] Logged in from keytab as kudu/127.28.154.131@KRBTEST.COM (short username kudu)
I20260811 09:07:43.303050 32072 webserver.cc:460] Webserver started at http://127.28.154.131:37267/ using document root <none> and password file <none>
I20260811 09:07:43.303965 32072 fs_manager.cc:359] Metadata directory not provided
I20260811 09:07:43.304111 32072 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:07:43.304477 32072 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:07:43.307511 32072 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439250192320-29290-0/minicluster-data/ts-2/data/instance:
uuid: "7264696b09d34a7580f9b3a89800d712"
format_stamp: "Formatted at 2026-08-11 09:07:43 on dist-test-slave-xt4k"
server_key: "8cd35624ca93c278c6896b8b70d8f8a7"
server_key_iv: "4f03b90fa255652459f91d34a21b84a4"
server_key_version: "encryptionkey@0"
I20260811 09:07:43.308359 32072 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439250192320-29290-0/minicluster-data/ts-2/wal/instance:
uuid: "7264696b09d34a7580f9b3a89800d712"
format_stamp: "Formatted at 2026-08-11 09:07:43 on dist-test-slave-xt4k"
server_key: "8cd35624ca93c278c6896b8b70d8f8a7"
server_key_iv: "4f03b90fa255652459f91d34a21b84a4"
server_key_version: "encryptionkey@0"
I20260811 09:07:43.308420 31819 heartbeater.cc:503] Master 127.28.154.190:45807 was elected leader, sending a full tablet report...
I20260811 09:07:43.314363 32072 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260811 09:07:43.318404 32105 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:07:43.320286 32072 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260811 09:07:43.320611 32072 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439250192320-29290-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439250192320-29290-0/minicluster-data/ts-2/wal
uuid: "7264696b09d34a7580f9b3a89800d712"
format_stamp: "Formatted at 2026-08-11 09:07:43 on dist-test-slave-xt4k"
server_key: "8cd35624ca93c278c6896b8b70d8f8a7"
server_key_iv: "4f03b90fa255652459f91d34a21b84a4"
server_key_version: "encryptionkey@0"
I20260811 09:07:43.320936 32072 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439250192320-29290-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439250192320-29290-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439250192320-29290-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:07:43.347468 32072 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:07:43.351755 32072 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:07:43.352027 32072 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:07:43.354848 32072 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:07:43.354936 32072 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:07:43.355069 32072 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:07:43.355145 32072 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 09:07:43 dist-test-slave-xt4k krb5kdc[31315](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439263, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.131@KRBTEST.COM for kudu/127.28.154.190@KRBTEST.COM
I20260811 09:07:43.380288 32118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:43.355490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.131:40773 (local address 127.28.154.190:45807)
0811 09:07:43.355952 (+ 462us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:43.355959 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:43.357385 (+ 1426us) server_negotiation.cc:407] Connection header received
0811 09:07:43.359464 (+ 2079us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:43.359470 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:43.359558 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:43.359652 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:43.361700 (+ 2048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:43.362981 (+ 1281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:43.366216 (+ 3235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:43.366797 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:43.366924 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:43.372073 (+ 5149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:07:43.372081 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:07:43.372086 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:07:43.372131 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:07:43.375865 (+ 3734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:43.376963 (+ 1098us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:43.376969 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:43.376973 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:43.377150 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:43.377822 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:43.377828 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:43.377831 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:43.378202 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:07:43.378358 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:43.379476 (+ 1118us) server_negotiation.cc:299] Negotiation successful
0811 09:07:43.379757 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":215,"threads_started":1}
I20260811 09:07:43.381372 32120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:43.356082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:45807 (local address 127.28.154.131:40773)
0811 09:07:43.357201 (+ 1119us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:43.357252 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 09:07:43.359195 (+ 1943us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:07:43.359903 (+ 708us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:07:43.359911 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:07:43.360625 (+ 714us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:07:43.361520 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:43.361548 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:43.363142 (+ 1594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:43.363148 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:43.366053 (+ 2905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:43.366067 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:43.367079 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:43.367089 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:43.367431 (+ 342us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:43.368128 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:07:43.368163 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:07:43.371760 (+ 3597us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:07:43.376016 (+ 4256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:07:43.376026 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:07:43.376045 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:07:43.376793 (+ 748us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:07:43.377314 (+ 521us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:07:43.377319 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:07:43.377323 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:07:43.377638 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:07:43.378343 (+ 705us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:07:43.378351 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:07:43.378922 (+ 571us) client_negotiation.cc:769] Sending connection context
0811 09:07:43.379344 (+ 422us) client_negotiation.cc:240] Negotiation successful
0811 09:07:43.380025 (+ 681us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":842,"thread_start_us":574,"threads_started":1}
I20260811 09:07:43.406961 32072 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.131:34691
I20260811 09:07:43.406994 32320 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.131:34691 every 8 connection(s)
I20260811 09:07:43.407866 32251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:43.390255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:45807 (local address 127.28.154.131:40477)
0811 09:07:43.391311 (+ 1056us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:43.391332 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 09:07:43.391574 (+ 242us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:07:43.392208 (+ 634us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:07:43.392215 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:07:43.392909 (+ 694us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:07:43.393745 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:43.393767 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:43.395249 (+ 1482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:43.395252 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:43.396752 (+ 1500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:43.396770 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:43.398122 (+ 1352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:43.398130 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:43.398336 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:43.399221 (+ 885us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:07:43.399250 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:07:43.400663 (+ 1413us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:07:43.404593 (+ 3930us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:07:43.404599 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:07:43.404603 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:07:43.405145 (+ 542us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:07:43.405654 (+ 509us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:07:43.405659 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:07:43.405664 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:07:43.405847 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:07:43.406654 (+ 807us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:07:43.406660 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:07:43.406911 (+ 251us) client_negotiation.cc:769] Sending connection context
0811 09:07:43.407064 (+ 153us) client_negotiation.cc:240] Negotiation successful
0811 09:07:43.407426 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":861,"thread_start_us":420,"threads_started":1}
I20260811 09:07:43.408646 32072 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439250192320-29290-0/minicluster-data/ts-2/data/info.pb
I20260811 09:07:43.408906 32118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:43.390209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.131:40477 (local address 127.28.154.190:45807)
0811 09:07:43.390408 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:43.390413 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:43.391433 (+ 1020us) server_negotiation.cc:407] Connection header received
0811 09:07:43.391749 (+ 316us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:43.391755 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:43.391846 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:43.391925 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:43.393954 (+ 2029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:43.395090 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:43.396938 (+ 1848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:43.397648 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:43.397753 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:43.400855 (+ 3102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:07:43.400862 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:07:43.400867 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:07:43.400910 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:07:43.404450 (+ 3540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:43.405282 (+ 832us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:43.405289 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:43.405292 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:43.405523 (+ 231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:43.405995 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:43.406001 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:43.406004 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:43.406519 (+ 515us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:07:43.406725 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:43.408317 (+ 1592us) server_negotiation.cc:299] Negotiation successful
0811 09:07:43.408587 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 09:07:43.411230 32322 heartbeater.cc:347] Connected to a master server at 127.28.154.190:45807
I20260811 09:07:43.412217 32322 heartbeater.cc:464] Registering TS with master...
I20260811 09:07:43.413128 32322 heartbeater.cc:511] Master 127.28.154.190:45807 requested a full tablet report, sending...
I20260811 09:07:43.414552 31403 ts_manager.cc:194] Registered new tserver with Master: 7264696b09d34a7580f9b3a89800d712 (127.28.154.131:34691)
I20260811 09:07:43.415342 31403 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.154.131@KRBTEST.COM'} at 127.28.154.131:40773
I20260811 09:07:43.416792 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 32072
I20260811 09:07:43.416957 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439250192320-29290-0/minicluster-data/ts-2/wal/instance
I20260811 09:07:43.417394 29290 external_mini_cluster.cc:1428] Setting key a6f97c0ee0b9e852eca341a15af2d28d
I20260811 09:07:43.420378 29290 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 09:07:43 dist-test-slave-xt4k krb5kdc[31315](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439263, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 09:07:43 dist-test-slave-xt4k krb5kdc[31315](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439263, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.28.154.190@KRBTEST.COM
I20260811 09:07:43.474614 32118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:43.458962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38224 (local address 127.28.154.190:45807)
0811 09:07:43.459212 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:43.459219 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:43.459362 (+ 143us) server_negotiation.cc:407] Connection header received
0811 09:07:43.459610 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:43.459615 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:43.459683 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:43.459774 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:43.461515 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:43.462272 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:43.463624 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:43.464173 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:43.464293 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:43.468698 (+ 4405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:07:43.468703 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:07:43.468707 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:07:43.468747 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:07:43.471399 (+ 2652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:43.472274 (+ 875us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:43.472279 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:43.472282 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:43.472509 (+ 227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:43.472949 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:43.472953 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:43.472955 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:43.473540 (+ 585us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:07:43.473796 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:43.474104 (+ 308us) server_negotiation.cc:299] Negotiation successful
0811 09:07:43.474287 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260811 09:07:43.482275 31403 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38224:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 09:07:43.489831 31403 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 09:07:43.515776 32357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:43.502898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.130:34727 (local address 127.0.0.1:47398)
0811 09:07:43.505781 (+ 2883us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:43.505799 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 09:07:43.505910 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:07:43.507579 (+ 1669us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:07:43.507587 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:07:43.507629 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:07:43.508455 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:43.508472 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:43.510464 (+ 1992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:43.510469 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:43.512924 (+ 2455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:43.512938 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:43.514722 (+ 1784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:43.514728 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:43.514958 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:43.515011 (+ 53us) client_negotiation.cc:769] Sending connection context
0811 09:07:43.515145 (+ 134us) client_negotiation.cc:240] Negotiation successful
0811 09:07:43.515330 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2751,"thread_start_us":3276,"threads_started":1}
I20260811 09:07:43.515856 32352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:43.503273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47398 (local address 127.28.154.130:34727)
0811 09:07:43.506885 (+ 3612us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:43.506893 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:43.506924 (+ 31us) server_negotiation.cc:407] Connection header received
0811 09:07:43.507029 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:43.507037 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:43.507317 (+ 280us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:43.508486 (+ 1169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:07:43.508748 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:43.510287 (+ 1539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:43.513125 (+ 2838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:43.514464 (+ 1339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:43.514768 (+ 304us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:43.514960 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:43.515216 (+ 256us) server_negotiation.cc:299] Negotiation successful
0811 09:07:43.515512 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3465,"thread_start_us":3331,"threads_started":1}
I20260811 09:07:43.516991 32347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:43.502692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.131:34691 (local address 127.0.0.1:60996)
0811 09:07:43.504883 (+ 2191us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:43.504936 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 09:07:43.505126 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:07:43.508508 (+ 3382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:07:43.508512 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:07:43.508539 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:07:43.509419 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:43.509432 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:43.511566 (+ 2134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:43.511570 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:43.513959 (+ 2389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:43.513975 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:43.515687 (+ 1712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:43.515694 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:43.515857 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:43.515900 (+ 43us) client_negotiation.cc:769] Sending connection context
0811 09:07:43.515962 (+ 62us) client_negotiation.cc:240] Negotiation successful
0811 09:07:43.516084 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1897,"mutex_wait_us":1,"thread_start_us":2824,"threads_started":1}
I20260811 09:07:43.517218 32355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:43.503378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60996 (local address 127.28.154.131:34691)
0811 09:07:43.507808 (+ 4430us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:43.507817 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:43.507855 (+ 38us) server_negotiation.cc:407] Connection header received
0811 09:07:43.507974 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:43.507982 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:43.508242 (+ 260us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:43.508800 (+ 558us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:07:43.509696 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:43.511430 (+ 1734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:43.514128 (+ 2698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:43.515452 (+ 1324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:43.516489 (+ 1037us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:43.516669 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:43.516746 (+ 77us) server_negotiation.cc:299] Negotiation successful
0811 09:07:43.516898 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4175,"thread_start_us":3928,"threads_started":1}
I20260811 09:07:43.518311 32348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:43.503174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.129:34213 (local address 127.0.0.1:44664)
0811 09:07:43.509961 (+ 6787us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:43.509979 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 09:07:43.510206 (+ 227us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:07:43.510960 (+ 754us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:07:43.510964 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:07:43.510988 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:07:43.511533 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:43.511548 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:43.513739 (+ 2191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:43.513744 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:43.515529 (+ 1785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:43.515545 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:43.517523 (+ 1978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:43.517528 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:43.517705 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:43.517739 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 09:07:43.517835 (+ 96us) client_negotiation.cc:240] Negotiation successful
0811 09:07:43.517957 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6639,"mutex_wait_us":22,"thread_start_us":6500,"threads_started":1}
I20260811 09:07:43.518353 32353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:43.503756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44664 (local address 127.28.154.129:34213)
0811 09:07:43.506440 (+ 2684us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:43.506451 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:43.510090 (+ 3639us) server_negotiation.cc:407] Connection header received
0811 09:07:43.510414 (+ 324us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:43.510423 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:43.510661 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:43.510822 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:07:43.511901 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:43.513552 (+ 1651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:43.515717 (+ 2165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:43.517116 (+ 1399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:43.517284 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:43.517442 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:43.517905 (+ 463us) server_negotiation.cc:299] Negotiation successful
0811 09:07:43.518072 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2499,"thread_start_us":2586,"threads_started":1}
I20260811 09:07:43.520022 31940 tablet_service.cc:1467] Processing CreateTablet for tablet c2cf506aa67a47e3952951bb83a03a27 (DEFAULT_TABLE table=test-table [id=6c4a13c723c540c9b171a423c1a8e03e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:07:43.521318 31940 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2cf506aa67a47e3952951bb83a03a27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:07:43.521152 32187 tablet_service.cc:1467] Processing CreateTablet for tablet c2cf506aa67a47e3952951bb83a03a27 (DEFAULT_TABLE table=test-table [id=6c4a13c723c540c9b171a423c1a8e03e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:07:43.521868 31664 tablet_service.cc:1467] Processing CreateTablet for tablet c2cf506aa67a47e3952951bb83a03a27 (DEFAULT_TABLE table=test-table [id=6c4a13c723c540c9b171a423c1a8e03e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:07:43.522631 32187 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2cf506aa67a47e3952951bb83a03a27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:07:43.523576 31664 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2cf506aa67a47e3952951bb83a03a27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:07:43.533422 32358 tablet_bootstrap.cc:492] T c2cf506aa67a47e3952951bb83a03a27 P 096a81e294374bb9aec1df09659f4d06: Bootstrap starting.
I20260811 09:07:43.537772 32358 tablet_bootstrap.cc:654] T c2cf506aa67a47e3952951bb83a03a27 P 096a81e294374bb9aec1df09659f4d06: Neither blocks nor log segments found. Creating new log.
I20260811 09:07:43.538638 32360 tablet_bootstrap.cc:492] T c2cf506aa67a47e3952951bb83a03a27 P 7264696b09d34a7580f9b3a89800d712: Bootstrap starting.
I20260811 09:07:43.539526 32363 tablet_bootstrap.cc:492] T c2cf506aa67a47e3952951bb83a03a27 P 44fb00a4d0c3453bb1a53ef91dc21027: Bootstrap starting.
I20260811 09:07:43.540063 32358 log.cc:824] T c2cf506aa67a47e3952951bb83a03a27 P 096a81e294374bb9aec1df09659f4d06: Log is configured to *not* fsync() on all Append() calls
I20260811 09:07:43.543073 32363 tablet_bootstrap.cc:654] T c2cf506aa67a47e3952951bb83a03a27 P 44fb00a4d0c3453bb1a53ef91dc21027: Neither blocks nor log segments found. Creating new log.
I20260811 09:07:43.543462 32358 tablet_bootstrap.cc:492] T c2cf506aa67a47e3952951bb83a03a27 P 096a81e294374bb9aec1df09659f4d06: No bootstrap required, opened a new log
I20260811 09:07:43.543495 32360 tablet_bootstrap.cc:654] T c2cf506aa67a47e3952951bb83a03a27 P 7264696b09d34a7580f9b3a89800d712: Neither blocks nor log segments found. Creating new log.
I20260811 09:07:43.543781 32358 ts_tablet_manager.cc:1397] T c2cf506aa67a47e3952951bb83a03a27 P 096a81e294374bb9aec1df09659f4d06: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.003s
I20260811 09:07:43.544216 32363 log.cc:824] T c2cf506aa67a47e3952951bb83a03a27 P 44fb00a4d0c3453bb1a53ef91dc21027: Log is configured to *not* fsync() on all Append() calls
I20260811 09:07:43.544943 32360 log.cc:824] T c2cf506aa67a47e3952951bb83a03a27 P 7264696b09d34a7580f9b3a89800d712: Log is configured to *not* fsync() on all Append() calls
I20260811 09:07:43.546999 32363 tablet_bootstrap.cc:492] T c2cf506aa67a47e3952951bb83a03a27 P 44fb00a4d0c3453bb1a53ef91dc21027: No bootstrap required, opened a new log
I20260811 09:07:43.547339 32363 ts_tablet_manager.cc:1397] T c2cf506aa67a47e3952951bb83a03a27 P 44fb00a4d0c3453bb1a53ef91dc21027: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260811 09:07:43.547816 32360 tablet_bootstrap.cc:492] T c2cf506aa67a47e3952951bb83a03a27 P 7264696b09d34a7580f9b3a89800d712: No bootstrap required, opened a new log
I20260811 09:07:43.548180 32360 ts_tablet_manager.cc:1397] T c2cf506aa67a47e3952951bb83a03a27 P 7264696b09d34a7580f9b3a89800d712: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260811 09:07:43.555219 32363 raft_consensus.cc:348] T c2cf506aa67a47e3952951bb83a03a27 P 44fb00a4d0c3453bb1a53ef91dc21027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7264696b09d34a7580f9b3a89800d712" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 34691 } } peers { permanent_uuid: "096a81e294374bb9aec1df09659f4d06" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34727 } } peers { permanent_uuid: "44fb00a4d0c3453bb1a53ef91dc21027" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 34213 } }
I20260811 09:07:43.555243 32358 raft_consensus.cc:348] T c2cf506aa67a47e3952951bb83a03a27 P 096a81e294374bb9aec1df09659f4d06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7264696b09d34a7580f9b3a89800d712" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 34691 } } peers { permanent_uuid: "096a81e294374bb9aec1df09659f4d06" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34727 } } peers { permanent_uuid: "44fb00a4d0c3453bb1a53ef91dc21027" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 34213 } }
I20260811 09:07:43.555657 32363 raft_consensus.cc:374] T c2cf506aa67a47e3952951bb83a03a27 P 44fb00a4d0c3453bb1a53ef91dc21027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:07:43.555684 32358 raft_consensus.cc:374] T c2cf506aa67a47e3952951bb83a03a27 P 096a81e294374bb9aec1df09659f4d06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:07:43.555771 32358 raft_consensus.cc:729] T c2cf506aa67a47e3952951bb83a03a27 P 096a81e294374bb9aec1df09659f4d06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 096a81e294374bb9aec1df09659f4d06, State: Initialized, Role: FOLLOWER
I20260811 09:07:43.555763 32363 raft_consensus.cc:729] T c2cf506aa67a47e3952951bb83a03a27 P 44fb00a4d0c3453bb1a53ef91dc21027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44fb00a4d0c3453bb1a53ef91dc21027, State: Initialized, Role: FOLLOWER
I20260811 09:07:43.556419 32363 consensus_queue.cc:260] T c2cf506aa67a47e3952951bb83a03a27 P 44fb00a4d0c3453bb1a53ef91dc21027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7264696b09d34a7580f9b3a89800d712" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 34691 } } peers { permanent_uuid: "096a81e294374bb9aec1df09659f4d06" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34727 } } peers { permanent_uuid: "44fb00a4d0c3453bb1a53ef91dc21027" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 34213 } }
I20260811 09:07:43.556460 32358 consensus_queue.cc:260] T c2cf506aa67a47e3952951bb83a03a27 P 096a81e294374bb9aec1df09659f4d06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7264696b09d34a7580f9b3a89800d712" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 34691 } } peers { permanent_uuid: "096a81e294374bb9aec1df09659f4d06" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34727 } } peers { permanent_uuid: "44fb00a4d0c3453bb1a53ef91dc21027" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 34213 } }
I20260811 09:07:43.559563 32066 heartbeater.cc:503] Master 127.28.154.190:45807 was elected leader, sending a full tablet report...
I20260811 09:07:43.559759 32363 ts_tablet_manager.cc:1428] T c2cf506aa67a47e3952951bb83a03a27 P 44fb00a4d0c3453bb1a53ef91dc21027: Time spent starting tablet: real 0.012s user 0.013s sys 0.000s
I20260811 09:07:43.560189 32358 ts_tablet_manager.cc:1428] T c2cf506aa67a47e3952951bb83a03a27 P 096a81e294374bb9aec1df09659f4d06: Time spent starting tablet: real 0.016s user 0.012s sys 0.005s
I20260811 09:07:43.560721 32360 raft_consensus.cc:348] T c2cf506aa67a47e3952951bb83a03a27 P 7264696b09d34a7580f9b3a89800d712 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7264696b09d34a7580f9b3a89800d712" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 34691 } } peers { permanent_uuid: "096a81e294374bb9aec1df09659f4d06" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34727 } } peers { permanent_uuid: "44fb00a4d0c3453bb1a53ef91dc21027" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 34213 } }
I20260811 09:07:43.561141 32360 raft_consensus.cc:374] T c2cf506aa67a47e3952951bb83a03a27 P 7264696b09d34a7580f9b3a89800d712 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:07:43.561249 32360 raft_consensus.cc:729] T c2cf506aa67a47e3952951bb83a03a27 P 7264696b09d34a7580f9b3a89800d712 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7264696b09d34a7580f9b3a89800d712, State: Initialized, Role: FOLLOWER
I20260811 09:07:43.561822 32360 consensus_queue.cc:260] T c2cf506aa67a47e3952951bb83a03a27 P 7264696b09d34a7580f9b3a89800d712 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7264696b09d34a7580f9b3a89800d712" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 34691 } } peers { permanent_uuid: "096a81e294374bb9aec1df09659f4d06" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34727 } } peers { permanent_uuid: "44fb00a4d0c3453bb1a53ef91dc21027" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 34213 } }
I20260811 09:07:43.563205 32322 heartbeater.cc:503] Master 127.28.154.190:45807 was elected leader, sending a full tablet report...
I20260811 09:07:43.563797 32360 ts_tablet_manager.cc:1428] T c2cf506aa67a47e3952951bb83a03a27 P 7264696b09d34a7580f9b3a89800d712: Time spent starting tablet: real 0.016s user 0.015s sys 0.000s
W20260811 09:07:43.579169 32067 tablet.cc:2369] T c2cf506aa67a47e3952951bb83a03a27 P 096a81e294374bb9aec1df09659f4d06: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 09:07:43.661746 32323 tablet.cc:2369] T c2cf506aa67a47e3952951bb83a03a27 P 7264696b09d34a7580f9b3a89800d712: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 09:07:43.789754 32367 raft_consensus.cc:482] T c2cf506aa67a47e3952951bb83a03a27 P 44fb00a4d0c3453bb1a53ef91dc21027 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:07:43.790316 32367 raft_consensus.cc:504] T c2cf506aa67a47e3952951bb83a03a27 P 44fb00a4d0c3453bb1a53ef91dc21027 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7264696b09d34a7580f9b3a89800d712" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 34691 } } peers { permanent_uuid: "096a81e294374bb9aec1df09659f4d06" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34727 } } peers { permanent_uuid: "44fb00a4d0c3453bb1a53ef91dc21027" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 34213 } }
I20260811 09:07:43.792824 32367 leader_election.cc:288] T c2cf506aa67a47e3952951bb83a03a27 P 44fb00a4d0c3453bb1a53ef91dc21027 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7264696b09d34a7580f9b3a89800d712 (127.28.154.131:34691), 096a81e294374bb9aec1df09659f4d06 (127.28.154.130:34727)
I20260811 09:07:43.800191 32352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:43.793284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.129:34373 (local address 127.28.154.130:34727)
0811 09:07:43.793540 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:43.793546 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:43.794580 (+ 1034us) server_negotiation.cc:407] Connection header received
0811 09:07:43.794738 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:43.794742 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:43.794802 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:43.794874 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:07:43.795564 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:43.796588 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:43.798496 (+ 1908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:43.799267 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:43.799438 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:43.799520 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:43.799793 (+ 273us) server_negotiation.cc:299] Negotiation successful
0811 09:07:43.799935 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110}
I20260811 09:07:43.800330 32374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:43.793348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.130:34727 (local address 127.28.154.129:34373)
0811 09:07:43.794477 (+ 1129us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:43.794494 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 09:07:43.794607 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:07:43.794905 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:07:43.794909 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:07:43.794934 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:07:43.795405 (+ 471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:43.795414 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:43.796738 (+ 1324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:43.796742 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:43.798330 (+ 1588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:43.798344 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:43.799494 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:43.799500 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:43.799639 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:43.799664 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 09:07:43.799752 (+ 88us) client_negotiation.cc:240] Negotiation successful
0811 09:07:43.799909 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":949,"mutex_wait_us":10,"thread_start_us":408,"threads_started":1}
I20260811 09:07:43.800606 32355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:43.793748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.129:58491 (local address 127.28.154.131:34691)
0811 09:07:43.794015 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:43.794022 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:43.794272 (+ 250us) server_negotiation.cc:407] Connection header received
0811 09:07:43.794537 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:43.794542 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:43.794608 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:43.794709 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:07:43.795504 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:43.796545 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:43.798502 (+ 1957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:43.799675 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:43.799831 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:43.799905 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:43.800198 (+ 293us) server_negotiation.cc:299] Negotiation successful
0811 09:07:43.800328 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260811 09:07:43.801072 32375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:43.793262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.131:34691 (local address 127.28.154.129:58491)
0811 09:07:43.794144 (+ 882us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:43.794166 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 09:07:43.794353 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:07:43.794765 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:07:43.794770 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:07:43.794800 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:07:43.795320 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:43.795331 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:43.796675 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:43.796681 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:43.798333 (+ 1652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:43.798346 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:43.799900 (+ 1554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:43.799906 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:43.800049 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:43.800071 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 09:07:43.800152 (+ 81us) client_negotiation.cc:240] Negotiation successful
0811 09:07:43.800804 (+ 652us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":724,"mutex_wait_us":34,"thread_start_us":874,"threads_started":1}
W20260811 09:07:43.801486 31820 tablet.cc:2369] T c2cf506aa67a47e3952951bb83a03a27 P 44fb00a4d0c3453bb1a53ef91dc21027: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 09:07:43.801450 31989 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2cf506aa67a47e3952951bb83a03a27" candidate_uuid: "44fb00a4d0c3453bb1a53ef91dc21027" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "096a81e294374bb9aec1df09659f4d06" is_pre_election: true
I20260811 09:07:43.802171 31989 raft_consensus.cc:2454] T c2cf506aa67a47e3952951bb83a03a27 P 096a81e294374bb9aec1df09659f4d06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 44fb00a4d0c3453bb1a53ef91dc21027 in term 0.
I20260811 09:07:43.802119 32227 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2cf506aa67a47e3952951bb83a03a27" candidate_uuid: "44fb00a4d0c3453bb1a53ef91dc21027" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7264696b09d34a7580f9b3a89800d712" is_pre_election: true
I20260811 09:07:43.802809 31544 leader_election.cc:302] T c2cf506aa67a47e3952951bb83a03a27 P 44fb00a4d0c3453bb1a53ef91dc21027 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 096a81e294374bb9aec1df09659f4d06, 44fb00a4d0c3453bb1a53ef91dc21027; no voters:
I20260811 09:07:43.802772 32227 raft_consensus.cc:2454] T c2cf506aa67a47e3952951bb83a03a27 P 7264696b09d34a7580f9b3a89800d712 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 44fb00a4d0c3453bb1a53ef91dc21027 in term 0.
I20260811 09:07:43.803184 32367 raft_consensus.cc:2781] T c2cf506aa67a47e3952951bb83a03a27 P 44fb00a4d0c3453bb1a53ef91dc21027 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 09:07:43.803412 32367 raft_consensus.cc:482] T c2cf506aa67a47e3952951bb83a03a27 P 44fb00a4d0c3453bb1a53ef91dc21027 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 09:07:43.803581 32367 raft_consensus.cc:3037] T c2cf506aa67a47e3952951bb83a03a27 P 44fb00a4d0c3453bb1a53ef91dc21027 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:07:43.805825 32367 raft_consensus.cc:504] T c2cf506aa67a47e3952951bb83a03a27 P 44fb00a4d0c3453bb1a53ef91dc21027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7264696b09d34a7580f9b3a89800d712" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 34691 } } peers { permanent_uuid: "096a81e294374bb9aec1df09659f4d06" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34727 } } peers { permanent_uuid: "44fb00a4d0c3453bb1a53ef91dc21027" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 34213 } }
I20260811 09:07:43.806406 32367 leader_election.cc:288] T c2cf506aa67a47e3952951bb83a03a27 P 44fb00a4d0c3453bb1a53ef91dc21027 [CANDIDATE]: Term 1 election: Requested vote from peers 7264696b09d34a7580f9b3a89800d712 (127.28.154.131:34691), 096a81e294374bb9aec1df09659f4d06 (127.28.154.130:34727)
I20260811 09:07:43.806937 32227 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2cf506aa67a47e3952951bb83a03a27" candidate_uuid: "44fb00a4d0c3453bb1a53ef91dc21027" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7264696b09d34a7580f9b3a89800d712"
I20260811 09:07:43.806939 31989 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2cf506aa67a47e3952951bb83a03a27" candidate_uuid: "44fb00a4d0c3453bb1a53ef91dc21027" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "096a81e294374bb9aec1df09659f4d06"
I20260811 09:07:43.807148 32227 raft_consensus.cc:3037] T c2cf506aa67a47e3952951bb83a03a27 P 7264696b09d34a7580f9b3a89800d712 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:07:43.807148 31989 raft_consensus.cc:3037] T c2cf506aa67a47e3952951bb83a03a27 P 096a81e294374bb9aec1df09659f4d06 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:07:43.809414 32227 raft_consensus.cc:2454] T c2cf506aa67a47e3952951bb83a03a27 P 7264696b09d34a7580f9b3a89800d712 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 44fb00a4d0c3453bb1a53ef91dc21027 in term 1.
I20260811 09:07:43.809772 31989 raft_consensus.cc:2454] T c2cf506aa67a47e3952951bb83a03a27 P 096a81e294374bb9aec1df09659f4d06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 44fb00a4d0c3453bb1a53ef91dc21027 in term 1.
I20260811 09:07:43.809967 31550 leader_election.cc:302] T c2cf506aa67a47e3952951bb83a03a27 P 44fb00a4d0c3453bb1a53ef91dc21027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 44fb00a4d0c3453bb1a53ef91dc21027, 7264696b09d34a7580f9b3a89800d712; no voters:
I20260811 09:07:43.810310 32367 raft_consensus.cc:2781] T c2cf506aa67a47e3952951bb83a03a27 P 44fb00a4d0c3453bb1a53ef91dc21027 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:07:43.810753 32367 raft_consensus.cc:686] T c2cf506aa67a47e3952951bb83a03a27 P 44fb00a4d0c3453bb1a53ef91dc21027 [term 1 LEADER]: Becoming Leader. State: Replica: 44fb00a4d0c3453bb1a53ef91dc21027, State: Running, Role: LEADER
I20260811 09:07:43.811240 32367 consensus_queue.cc:237] T c2cf506aa67a47e3952951bb83a03a27 P 44fb00a4d0c3453bb1a53ef91dc21027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7264696b09d34a7580f9b3a89800d712" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 34691 } } peers { permanent_uuid: "096a81e294374bb9aec1df09659f4d06" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34727 } } peers { permanent_uuid: "44fb00a4d0c3453bb1a53ef91dc21027" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 34213 } }
I20260811 09:07:43.815943 31400 catalog_manager.cc:5495] T c2cf506aa67a47e3952951bb83a03a27 P 44fb00a4d0c3453bb1a53ef91dc21027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 44fb00a4d0c3453bb1a53ef91dc21027 (127.28.154.129). New cstate: current_term: 1 leader_uuid: "44fb00a4d0c3453bb1a53ef91dc21027" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7264696b09d34a7580f9b3a89800d712" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 34691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "096a81e294374bb9aec1df09659f4d06" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "44fb00a4d0c3453bb1a53ef91dc21027" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 34213 } health_report { overall_health: HEALTHY } } }
I20260811 09:07:43.910580 32353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:43.905140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44666 (local address 127.28.154.129:34213)
0811 09:07:43.905388 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:43.905393 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:43.905412 (+ 19us) server_negotiation.cc:407] Connection header received
0811 09:07:43.905661 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:43.905666 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:43.905750 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:43.905869 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:07:43.906270 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:43.906959 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:43.908314 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:43.908781 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:43.908909 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:43.909339 (+ 430us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:07:43.909966 (+ 627us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:07:43.910090 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:43.910265 (+ 175us) server_negotiation.cc:299] Negotiation successful
0811 09:07:43.910341 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260811 09:07:43.922783 31989 raft_consensus.cc:1261] T c2cf506aa67a47e3952951bb83a03a27 P 096a81e294374bb9aec1df09659f4d06 [term 1 FOLLOWER]: Refusing update from remote peer 44fb00a4d0c3453bb1a53ef91dc21027: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:07:43.924031 32377 consensus_queue.cc:1035] T c2cf506aa67a47e3952951bb83a03a27 P 44fb00a4d0c3453bb1a53ef91dc21027 [LEADER]: Connected to new peer: Peer: permanent_uuid: "096a81e294374bb9aec1df09659f4d06" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:07:43.924641 32227 raft_consensus.cc:1261] T c2cf506aa67a47e3952951bb83a03a27 P 7264696b09d34a7580f9b3a89800d712 [term 1 FOLLOWER]: Refusing update from remote peer 44fb00a4d0c3453bb1a53ef91dc21027: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:07:43.925474 32377 consensus_queue.cc:1035] T c2cf506aa67a47e3952951bb83a03a27 P 44fb00a4d0c3453bb1a53ef91dc21027 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7264696b09d34a7580f9b3a89800d712" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 34691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:07:43.960116 31403 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38224:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:07:43.960625 31403 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38224:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:07:43.965363 31403 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f6cfaebcd30e476f93f1c0f1154871c7: Sending DeleteTablet for 3 replicas of tablet c2cf506aa67a47e3952951bb83a03a27
I20260811 09:07:43.967017 31940 tablet_service.cc:1514] Processing DeleteTablet for tablet c2cf506aa67a47e3952951bb83a03a27 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:07:43 UTC) from {username='kudu', principal='kudu/127.28.154.190@KRBTEST.COM'} at 127.0.0.1:47398
I20260811 09:07:43.966974 32187 tablet_service.cc:1514] Processing DeleteTablet for tablet c2cf506aa67a47e3952951bb83a03a27 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:07:43 UTC) from {username='kudu', principal='kudu/127.28.154.190@KRBTEST.COM'} at 127.0.0.1:60996
I20260811 09:07:43.967427 31664 tablet_service.cc:1514] Processing DeleteTablet for tablet c2cf506aa67a47e3952951bb83a03a27 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:07:43 UTC) from {username='kudu', principal='kudu/127.28.154.190@KRBTEST.COM'} at 127.0.0.1:44664
I20260811 09:07:43.972126 31403 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38224:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 09:07:43.972915 31403 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 09:07:43.976116 32407 tablet_replica.cc:330] T c2cf506aa67a47e3952951bb83a03a27 P 096a81e294374bb9aec1df09659f4d06: stopping tablet replica
I20260811 09:07:43.976816 32407 raft_consensus.cc:2229] T c2cf506aa67a47e3952951bb83a03a27 P 096a81e294374bb9aec1df09659f4d06 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:07:43.977380 32407 raft_consensus.cc:2258] T c2cf506aa67a47e3952951bb83a03a27 P 096a81e294374bb9aec1df09659f4d06 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:07:43.978260 32402 tablet_replica.cc:330] T c2cf506aa67a47e3952951bb83a03a27 P 7264696b09d34a7580f9b3a89800d712: stopping tablet replica
I20260811 09:07:43.978598 32405 tablet_replica.cc:330] T c2cf506aa67a47e3952951bb83a03a27 P 44fb00a4d0c3453bb1a53ef91dc21027: stopping tablet replica
I20260811 09:07:43.978693 32402 raft_consensus.cc:2229] T c2cf506aa67a47e3952951bb83a03a27 P 7264696b09d34a7580f9b3a89800d712 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:07:43.979187 32402 raft_consensus.cc:2258] T c2cf506aa67a47e3952951bb83a03a27 P 7264696b09d34a7580f9b3a89800d712 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:07:43.979138 32405 raft_consensus.cc:2229] T c2cf506aa67a47e3952951bb83a03a27 P 44fb00a4d0c3453bb1a53ef91dc21027 [term 1 LEADER]: Raft consensus shutting down.
I20260811 09:07:43.980294 32405 raft_consensus.cc:2258] T c2cf506aa67a47e3952951bb83a03a27 P 44fb00a4d0c3453bb1a53ef91dc21027 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:07:43.982326 31940 tablet_service.cc:1467] Processing CreateTablet for tablet 4f59ca5dde514bf88072bb7d7aa3aa42 (DEFAULT_TABLE table=test-table [id=747620bab7b74dbabdb7f8c61012397a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:07:43.982326 32187 tablet_service.cc:1467] Processing CreateTablet for tablet 4f59ca5dde514bf88072bb7d7aa3aa42 (DEFAULT_TABLE table=test-table [id=747620bab7b74dbabdb7f8c61012397a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:07:43.982808 31940 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f59ca5dde514bf88072bb7d7aa3aa42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:07:43.982818 32187 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f59ca5dde514bf88072bb7d7aa3aa42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:07:43.983557 31664 tablet_service.cc:1467] Processing CreateTablet for tablet 4f59ca5dde514bf88072bb7d7aa3aa42 (DEFAULT_TABLE table=test-table [id=747620bab7b74dbabdb7f8c61012397a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:07:43.984081 31664 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f59ca5dde514bf88072bb7d7aa3aa42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:07:43.989197 32360 tablet_bootstrap.cc:492] T 4f59ca5dde514bf88072bb7d7aa3aa42 P 7264696b09d34a7580f9b3a89800d712: Bootstrap starting.
I20260811 09:07:43.990821 32360 tablet_bootstrap.cc:654] T 4f59ca5dde514bf88072bb7d7aa3aa42 P 7264696b09d34a7580f9b3a89800d712: Neither blocks nor log segments found. Creating new log.
I20260811 09:07:43.992169 32407 ts_tablet_manager.cc:1905] T c2cf506aa67a47e3952951bb83a03a27 P 096a81e294374bb9aec1df09659f4d06: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:07:43.993021 32358 tablet_bootstrap.cc:492] T 4f59ca5dde514bf88072bb7d7aa3aa42 P 096a81e294374bb9aec1df09659f4d06: Bootstrap starting.
I20260811 09:07:43.994614 32358 tablet_bootstrap.cc:654] T 4f59ca5dde514bf88072bb7d7aa3aa42 P 096a81e294374bb9aec1df09659f4d06: Neither blocks nor log segments found. Creating new log.
I20260811 09:07:43.998353 32363 tablet_bootstrap.cc:492] T 4f59ca5dde514bf88072bb7d7aa3aa42 P 44fb00a4d0c3453bb1a53ef91dc21027: Bootstrap starting.
I20260811 09:07:44.000051 32363 tablet_bootstrap.cc:654] T 4f59ca5dde514bf88072bb7d7aa3aa42 P 44fb00a4d0c3453bb1a53ef91dc21027: Neither blocks nor log segments found. Creating new log.
I20260811 09:07:44.002260 32402 ts_tablet_manager.cc:1905] T c2cf506aa67a47e3952951bb83a03a27 P 7264696b09d34a7580f9b3a89800d712: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:07:44.004889 32405 ts_tablet_manager.cc:1905] T c2cf506aa67a47e3952951bb83a03a27 P 44fb00a4d0c3453bb1a53ef91dc21027: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:07:44.006721 32360 tablet_bootstrap.cc:492] T 4f59ca5dde514bf88072bb7d7aa3aa42 P 7264696b09d34a7580f9b3a89800d712: No bootstrap required, opened a new log
I20260811 09:07:44.006893 32360 ts_tablet_manager.cc:1397] T 4f59ca5dde514bf88072bb7d7aa3aa42 P 7264696b09d34a7580f9b3a89800d712: Time spent bootstrapping tablet: real 0.018s user 0.004s sys 0.005s
I20260811 09:07:44.007717 32360 raft_consensus.cc:348] T 4f59ca5dde514bf88072bb7d7aa3aa42 P 7264696b09d34a7580f9b3a89800d712 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44fb00a4d0c3453bb1a53ef91dc21027" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 34213 } } peers { permanent_uuid: "096a81e294374bb9aec1df09659f4d06" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34727 } } peers { permanent_uuid: "7264696b09d34a7580f9b3a89800d712" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 34691 } }
I20260811 09:07:44.007963 32360 raft_consensus.cc:374] T 4f59ca5dde514bf88072bb7d7aa3aa42 P 7264696b09d34a7580f9b3a89800d712 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:07:44.008059 32360 raft_consensus.cc:729] T 4f59ca5dde514bf88072bb7d7aa3aa42 P 7264696b09d34a7580f9b3a89800d712 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7264696b09d34a7580f9b3a89800d712, State: Initialized, Role: FOLLOWER
I20260811 09:07:44.009805 32358 tablet_bootstrap.cc:492] T 4f59ca5dde514bf88072bb7d7aa3aa42 P 096a81e294374bb9aec1df09659f4d06: No bootstrap required, opened a new log
I20260811 09:07:44.009981 32358 ts_tablet_manager.cc:1397] T 4f59ca5dde514bf88072bb7d7aa3aa42 P 096a81e294374bb9aec1df09659f4d06: Time spent bootstrapping tablet: real 0.017s user 0.012s sys 0.003s
I20260811 09:07:44.010376 32360 consensus_queue.cc:260] T 4f59ca5dde514bf88072bb7d7aa3aa42 P 7264696b09d34a7580f9b3a89800d712 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44fb00a4d0c3453bb1a53ef91dc21027" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 34213 } } peers { permanent_uuid: "096a81e294374bb9aec1df09659f4d06" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34727 } } peers { permanent_uuid: "7264696b09d34a7580f9b3a89800d712" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 34691 } }
I20260811 09:07:44.010810 32358 raft_consensus.cc:348] T 4f59ca5dde514bf88072bb7d7aa3aa42 P 096a81e294374bb9aec1df09659f4d06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44fb00a4d0c3453bb1a53ef91dc21027" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 34213 } } peers { permanent_uuid: "096a81e294374bb9aec1df09659f4d06" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34727 } } peers { permanent_uuid: "7264696b09d34a7580f9b3a89800d712" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 34691 } }
I20260811 09:07:44.011050 32358 raft_consensus.cc:374] T 4f59ca5dde514bf88072bb7d7aa3aa42 P 096a81e294374bb9aec1df09659f4d06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:07:44.011157 32358 raft_consensus.cc:729] T 4f59ca5dde514bf88072bb7d7aa3aa42 P 096a81e294374bb9aec1df09659f4d06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 096a81e294374bb9aec1df09659f4d06, State: Initialized, Role: FOLLOWER
I20260811 09:07:44.014166 32360 ts_tablet_manager.cc:1428] T 4f59ca5dde514bf88072bb7d7aa3aa42 P 7264696b09d34a7580f9b3a89800d712: Time spent starting tablet: real 0.007s user 0.002s sys 0.000s
I20260811 09:07:44.014562 32402 ts_tablet_manager.cc:1918] T c2cf506aa67a47e3952951bb83a03a27 P 7264696b09d34a7580f9b3a89800d712: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:07:44.014842 32402 log.cc:1196] T c2cf506aa67a47e3952951bb83a03a27 P 7264696b09d34a7580f9b3a89800d712: Deleting WAL directory at /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439250192320-29290-0/minicluster-data/ts-2/wal/wals/c2cf506aa67a47e3952951bb83a03a27
I20260811 09:07:44.015419 32402 ts_tablet_manager.cc:1939] T c2cf506aa67a47e3952951bb83a03a27 P 7264696b09d34a7580f9b3a89800d712: Deleting consensus metadata
I20260811 09:07:44.015784 32405 ts_tablet_manager.cc:1918] T c2cf506aa67a47e3952951bb83a03a27 P 44fb00a4d0c3453bb1a53ef91dc21027: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:07:44.015969 32405 log.cc:1196] T c2cf506aa67a47e3952951bb83a03a27 P 44fb00a4d0c3453bb1a53ef91dc21027: Deleting WAL directory at /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439250192320-29290-0/minicluster-data/ts-0/wal/wals/c2cf506aa67a47e3952951bb83a03a27
I20260811 09:07:44.016738 32358 consensus_queue.cc:260] T 4f59ca5dde514bf88072bb7d7aa3aa42 P 096a81e294374bb9aec1df09659f4d06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44fb00a4d0c3453bb1a53ef91dc21027" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 34213 } } peers { permanent_uuid: "096a81e294374bb9aec1df09659f4d06" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34727 } } peers { permanent_uuid: "7264696b09d34a7580f9b3a89800d712" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 34691 } }
I20260811 09:07:44.017799 31386 catalog_manager.cc:4841] TS 7264696b09d34a7580f9b3a89800d712 (127.28.154.131:34691): tablet c2cf506aa67a47e3952951bb83a03a27 (table test-table [id=6c4a13c723c540c9b171a423c1a8e03e]) successfully deleted
I20260811 09:07:44.018545 32358 ts_tablet_manager.cc:1428] T 4f59ca5dde514bf88072bb7d7aa3aa42 P 096a81e294374bb9aec1df09659f4d06: Time spent starting tablet: real 0.008s user 0.002s sys 0.001s
I20260811 09:07:44.018728 32407 ts_tablet_manager.cc:1918] T c2cf506aa67a47e3952951bb83a03a27 P 096a81e294374bb9aec1df09659f4d06: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:07:44.018894 32407 log.cc:1196] T c2cf506aa67a47e3952951bb83a03a27 P 096a81e294374bb9aec1df09659f4d06: Deleting WAL directory at /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439250192320-29290-0/minicluster-data/ts-1/wal/wals/c2cf506aa67a47e3952951bb83a03a27
I20260811 09:07:44.019454 32407 ts_tablet_manager.cc:1939] T c2cf506aa67a47e3952951bb83a03a27 P 096a81e294374bb9aec1df09659f4d06: Deleting consensus metadata
I20260811 09:07:44.021350 31379 catalog_manager.cc:4841] TS 096a81e294374bb9aec1df09659f4d06 (127.28.154.130:34727): tablet c2cf506aa67a47e3952951bb83a03a27 (table test-table [id=6c4a13c723c540c9b171a423c1a8e03e]) successfully deleted
I20260811 09:07:44.021919 32405 ts_tablet_manager.cc:1939] T c2cf506aa67a47e3952951bb83a03a27 P 44fb00a4d0c3453bb1a53ef91dc21027: Deleting consensus metadata
I20260811 09:07:44.023279 32363 tablet_bootstrap.cc:492] T 4f59ca5dde514bf88072bb7d7aa3aa42 P 44fb00a4d0c3453bb1a53ef91dc21027: No bootstrap required, opened a new log
I20260811 09:07:44.023459 32363 ts_tablet_manager.cc:1397] T 4f59ca5dde514bf88072bb7d7aa3aa42 P 44fb00a4d0c3453bb1a53ef91dc21027: Time spent bootstrapping tablet: real 0.025s user 0.005s sys 0.009s
I20260811 09:07:44.023968 31383 catalog_manager.cc:4841] TS 44fb00a4d0c3453bb1a53ef91dc21027 (127.28.154.129:34213): tablet c2cf506aa67a47e3952951bb83a03a27 (table test-table [id=6c4a13c723c540c9b171a423c1a8e03e]) successfully deleted
I20260811 09:07:44.024397 32363 raft_consensus.cc:348] T 4f59ca5dde514bf88072bb7d7aa3aa42 P 44fb00a4d0c3453bb1a53ef91dc21027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44fb00a4d0c3453bb1a53ef91dc21027" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 34213 } } peers { permanent_uuid: "096a81e294374bb9aec1df09659f4d06" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34727 } } peers { permanent_uuid: "7264696b09d34a7580f9b3a89800d712" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 34691 } }
I20260811 09:07:44.024650 32363 raft_consensus.cc:374] T 4f59ca5dde514bf88072bb7d7aa3aa42 P 44fb00a4d0c3453bb1a53ef91dc21027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:07:44.024755 32363 raft_consensus.cc:729] T 4f59ca5dde514bf88072bb7d7aa3aa42 P 44fb00a4d0c3453bb1a53ef91dc21027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44fb00a4d0c3453bb1a53ef91dc21027, State: Initialized, Role: FOLLOWER
I20260811 09:07:44.025044 32363 consensus_queue.cc:260] T 4f59ca5dde514bf88072bb7d7aa3aa42 P 44fb00a4d0c3453bb1a53ef91dc21027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44fb00a4d0c3453bb1a53ef91dc21027" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 34213 } } peers { permanent_uuid: "096a81e294374bb9aec1df09659f4d06" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34727 } } peers { permanent_uuid: "7264696b09d34a7580f9b3a89800d712" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 34691 } }
I20260811 09:07:44.025692 32363 ts_tablet_manager.cc:1428] T 4f59ca5dde514bf88072bb7d7aa3aa42 P 44fb00a4d0c3453bb1a53ef91dc21027: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260811 09:07:44.191936 32367 raft_consensus.cc:482] T 4f59ca5dde514bf88072bb7d7aa3aa42 P 44fb00a4d0c3453bb1a53ef91dc21027 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:07:44.192210 32367 raft_consensus.cc:504] T 4f59ca5dde514bf88072bb7d7aa3aa42 P 44fb00a4d0c3453bb1a53ef91dc21027 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44fb00a4d0c3453bb1a53ef91dc21027" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 34213 } } peers { permanent_uuid: "096a81e294374bb9aec1df09659f4d06" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34727 } } peers { permanent_uuid: "7264696b09d34a7580f9b3a89800d712" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 34691 } }
I20260811 09:07:44.192974 32367 leader_election.cc:288] T 4f59ca5dde514bf88072bb7d7aa3aa42 P 44fb00a4d0c3453bb1a53ef91dc21027 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 096a81e294374bb9aec1df09659f4d06 (127.28.154.130:34727), 7264696b09d34a7580f9b3a89800d712 (127.28.154.131:34691)
I20260811 09:07:44.193605 32227 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f59ca5dde514bf88072bb7d7aa3aa42" candidate_uuid: "44fb00a4d0c3453bb1a53ef91dc21027" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7264696b09d34a7580f9b3a89800d712" is_pre_election: true
I20260811 09:07:44.193642 31987 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f59ca5dde514bf88072bb7d7aa3aa42" candidate_uuid: "44fb00a4d0c3453bb1a53ef91dc21027" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "096a81e294374bb9aec1df09659f4d06" is_pre_election: true
I20260811 09:07:44.193898 32227 raft_consensus.cc:2454] T 4f59ca5dde514bf88072bb7d7aa3aa42 P 7264696b09d34a7580f9b3a89800d712 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 44fb00a4d0c3453bb1a53ef91dc21027 in term 0.
I20260811 09:07:44.193969 31987 raft_consensus.cc:2454] T 4f59ca5dde514bf88072bb7d7aa3aa42 P 096a81e294374bb9aec1df09659f4d06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 44fb00a4d0c3453bb1a53ef91dc21027 in term 0.
I20260811 09:07:44.194404 31550 leader_election.cc:302] T 4f59ca5dde514bf88072bb7d7aa3aa42 P 44fb00a4d0c3453bb1a53ef91dc21027 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 44fb00a4d0c3453bb1a53ef91dc21027, 7264696b09d34a7580f9b3a89800d712; no voters:
I20260811 09:07:44.194752 32367 raft_consensus.cc:2781] T 4f59ca5dde514bf88072bb7d7aa3aa42 P 44fb00a4d0c3453bb1a53ef91dc21027 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 09:07:44.194855 32367 raft_consensus.cc:482] T 4f59ca5dde514bf88072bb7d7aa3aa42 P 44fb00a4d0c3453bb1a53ef91dc21027 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 09:07:44.194977 32367 raft_consensus.cc:3037] T 4f59ca5dde514bf88072bb7d7aa3aa42 P 44fb00a4d0c3453bb1a53ef91dc21027 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:07:44.197273 32367 raft_consensus.cc:504] T 4f59ca5dde514bf88072bb7d7aa3aa42 P 44fb00a4d0c3453bb1a53ef91dc21027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44fb00a4d0c3453bb1a53ef91dc21027" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 34213 } } peers { permanent_uuid: "096a81e294374bb9aec1df09659f4d06" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34727 } } peers { permanent_uuid: "7264696b09d34a7580f9b3a89800d712" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 34691 } }
I20260811 09:07:44.197898 32367 leader_election.cc:288] T 4f59ca5dde514bf88072bb7d7aa3aa42 P 44fb00a4d0c3453bb1a53ef91dc21027 [CANDIDATE]: Term 1 election: Requested vote from peers 096a81e294374bb9aec1df09659f4d06 (127.28.154.130:34727), 7264696b09d34a7580f9b3a89800d712 (127.28.154.131:34691)
I20260811 09:07:44.198505 31987 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f59ca5dde514bf88072bb7d7aa3aa42" candidate_uuid: "44fb00a4d0c3453bb1a53ef91dc21027" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "096a81e294374bb9aec1df09659f4d06"
I20260811 09:07:44.198493 32227 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f59ca5dde514bf88072bb7d7aa3aa42" candidate_uuid: "44fb00a4d0c3453bb1a53ef91dc21027" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7264696b09d34a7580f9b3a89800d712"
I20260811 09:07:44.198722 31987 raft_consensus.cc:3037] T 4f59ca5dde514bf88072bb7d7aa3aa42 P 096a81e294374bb9aec1df09659f4d06 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:07:44.198724 32227 raft_consensus.cc:3037] T 4f59ca5dde514bf88072bb7d7aa3aa42 P 7264696b09d34a7580f9b3a89800d712 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:07:44.201709 31987 raft_consensus.cc:2454] T 4f59ca5dde514bf88072bb7d7aa3aa42 P 096a81e294374bb9aec1df09659f4d06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 44fb00a4d0c3453bb1a53ef91dc21027 in term 1.
I20260811 09:07:44.201709 32227 raft_consensus.cc:2454] T 4f59ca5dde514bf88072bb7d7aa3aa42 P 7264696b09d34a7580f9b3a89800d712 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 44fb00a4d0c3453bb1a53ef91dc21027 in term 1.
I20260811 09:07:44.202206 31544 leader_election.cc:302] T 4f59ca5dde514bf88072bb7d7aa3aa42 P 44fb00a4d0c3453bb1a53ef91dc21027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 096a81e294374bb9aec1df09659f4d06, 44fb00a4d0c3453bb1a53ef91dc21027; no voters:
I20260811 09:07:44.202574 32367 raft_consensus.cc:2781] T 4f59ca5dde514bf88072bb7d7aa3aa42 P 44fb00a4d0c3453bb1a53ef91dc21027 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:07:44.202775 32367 raft_consensus.cc:686] T 4f59ca5dde514bf88072bb7d7aa3aa42 P 44fb00a4d0c3453bb1a53ef91dc21027 [term 1 LEADER]: Becoming Leader. State: Replica: 44fb00a4d0c3453bb1a53ef91dc21027, State: Running, Role: LEADER
I20260811 09:07:44.203076 32367 consensus_queue.cc:237] T 4f59ca5dde514bf88072bb7d7aa3aa42 P 44fb00a4d0c3453bb1a53ef91dc21027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44fb00a4d0c3453bb1a53ef91dc21027" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 34213 } } peers { permanent_uuid: "096a81e294374bb9aec1df09659f4d06" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34727 } } peers { permanent_uuid: "7264696b09d34a7580f9b3a89800d712" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 34691 } }
I20260811 09:07:44.206432 31403 catalog_manager.cc:5495] T 4f59ca5dde514bf88072bb7d7aa3aa42 P 44fb00a4d0c3453bb1a53ef91dc21027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 44fb00a4d0c3453bb1a53ef91dc21027 (127.28.154.129). New cstate: current_term: 1 leader_uuid: "44fb00a4d0c3453bb1a53ef91dc21027" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44fb00a4d0c3453bb1a53ef91dc21027" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 34213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "096a81e294374bb9aec1df09659f4d06" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7264696b09d34a7580f9b3a89800d712" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 34691 } health_report { overall_health: UNKNOWN } } }
Aug 11 09:07:44 dist-test-slave-xt4k krb5kdc[31315](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439261, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.190@KRBTEST.COM for kudu/127.28.154.190@KRBTEST.COM
I20260811 09:07:44.271584 32427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:44.254934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:45807 (local address 127.0.0.1:38238)
0811 09:07:44.256149 (+ 1215us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:44.256172 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 09:07:44.256386 (+ 214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:07:44.256763 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:07:44.256768 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:07:44.257753 (+ 985us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:07:44.258036 (+ 283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:44.258053 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:44.259166 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:44.259172 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:44.260681 (+ 1509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:44.260697 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:44.261463 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:44.261471 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:44.261643 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:44.262430 (+ 787us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:07:44.262463 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:07:44.265512 (+ 3049us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:07:44.268692 (+ 3180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:07:44.268707 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:07:44.268715 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:07:44.269323 (+ 608us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:07:44.269682 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:07:44.269687 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:07:44.269691 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:07:44.269860 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:07:44.270396 (+ 536us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:07:44.270405 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:07:44.270629 (+ 224us) client_negotiation.cc:769] Sending connection context
0811 09:07:44.270818 (+ 189us) client_negotiation.cc:240] Negotiation successful
0811 09:07:44.271144 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1037,"thread_start_us":824,"threads_started":1}
I20260811 09:07:44.271672 32428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:44.254978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38238 (local address 127.28.154.190:45807)
0811 09:07:44.255801 (+ 823us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:44.255811 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:44.256289 (+ 478us) server_negotiation.cc:407] Connection header received
0811 09:07:44.256514 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:44.256518 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:44.256589 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:44.256682 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:44.258213 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:44.259010 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:44.260825 (+ 1815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:44.261283 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:44.261380 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:44.265781 (+ 4401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:07:44.265799 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:07:44.265806 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:07:44.265847 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:07:44.268563 (+ 2716us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:44.269441 (+ 878us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:44.269445 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:44.269447 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:44.269572 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:44.269980 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:44.269983 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:44.269985 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:44.270265 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:07:44.270383 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:44.270939 (+ 556us) server_negotiation.cc:299] Negotiation successful
0811 09:07:44.271321 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":157,"server-negotiator.queue_time_us":644,"thread_start_us":503,"threads_started":1}
I20260811 09:07:44.277633 31403 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.28.154.190@KRBTEST.COM'} at 127.0.0.1:38238:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 09:07:44.279280 31403 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 09:07:44.287101 31664 tablet_service.cc:1467] Processing CreateTablet for tablet dbb51c1bae6c408f9286965085478649 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=7b01b8d36b924f78aba0ad09c76ad66b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 09:07:44.287470 32187 tablet_service.cc:1467] Processing CreateTablet for tablet dbb51c1bae6c408f9286965085478649 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=7b01b8d36b924f78aba0ad09c76ad66b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 09:07:44.287470 31940 tablet_service.cc:1467] Processing CreateTablet for tablet dbb51c1bae6c408f9286965085478649 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=7b01b8d36b924f78aba0ad09c76ad66b]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 09:07:44.287792 31664 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dbb51c1bae6c408f9286965085478649. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:07:44.288021 31940 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dbb51c1bae6c408f9286965085478649. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:07:44.288021 32187 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dbb51c1bae6c408f9286965085478649. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:07:44.297431 32358 tablet_bootstrap.cc:492] T dbb51c1bae6c408f9286965085478649 P 096a81e294374bb9aec1df09659f4d06: Bootstrap starting.
I20260811 09:07:44.297533 32363 tablet_bootstrap.cc:492] T dbb51c1bae6c408f9286965085478649 P 44fb00a4d0c3453bb1a53ef91dc21027: Bootstrap starting.
I20260811 09:07:44.299229 32363 tablet_bootstrap.cc:654] T dbb51c1bae6c408f9286965085478649 P 44fb00a4d0c3453bb1a53ef91dc21027: Neither blocks nor log segments found. Creating new log.
I20260811 09:07:44.299227 32358 tablet_bootstrap.cc:654] T dbb51c1bae6c408f9286965085478649 P 096a81e294374bb9aec1df09659f4d06: Neither blocks nor log segments found. Creating new log.
I20260811 09:07:44.300299 32360 tablet_bootstrap.cc:492] T dbb51c1bae6c408f9286965085478649 P 7264696b09d34a7580f9b3a89800d712: Bootstrap starting.
I20260811 09:07:44.301847 32360 tablet_bootstrap.cc:654] T dbb51c1bae6c408f9286965085478649 P 7264696b09d34a7580f9b3a89800d712: Neither blocks nor log segments found. Creating new log.
I20260811 09:07:44.309823 32363 tablet_bootstrap.cc:492] T dbb51c1bae6c408f9286965085478649 P 44fb00a4d0c3453bb1a53ef91dc21027: No bootstrap required, opened a new log
I20260811 09:07:44.310026 32363 ts_tablet_manager.cc:1397] T dbb51c1bae6c408f9286965085478649 P 44fb00a4d0c3453bb1a53ef91dc21027: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.008s
I20260811 09:07:44.310760 32363 raft_consensus.cc:348] T dbb51c1bae6c408f9286965085478649 P 44fb00a4d0c3453bb1a53ef91dc21027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44fb00a4d0c3453bb1a53ef91dc21027" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 34213 } } peers { permanent_uuid: "096a81e294374bb9aec1df09659f4d06" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34727 } } peers { permanent_uuid: "7264696b09d34a7580f9b3a89800d712" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 34691 } }
I20260811 09:07:44.311020 32363 raft_consensus.cc:374] T dbb51c1bae6c408f9286965085478649 P 44fb00a4d0c3453bb1a53ef91dc21027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:07:44.311095 32363 raft_consensus.cc:729] T dbb51c1bae6c408f9286965085478649 P 44fb00a4d0c3453bb1a53ef91dc21027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44fb00a4d0c3453bb1a53ef91dc21027, State: Initialized, Role: FOLLOWER
I20260811 09:07:44.311388 32363 consensus_queue.cc:260] T dbb51c1bae6c408f9286965085478649 P 44fb00a4d0c3453bb1a53ef91dc21027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44fb00a4d0c3453bb1a53ef91dc21027" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 34213 } } peers { permanent_uuid: "096a81e294374bb9aec1df09659f4d06" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34727 } } peers { permanent_uuid: "7264696b09d34a7580f9b3a89800d712" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 34691 } }
I20260811 09:07:44.311932 32367 tablet_replica.cc:439] T dbb51c1bae6c408f9286965085478649 P 44fb00a4d0c3453bb1a53ef91dc21027: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44fb00a4d0c3453bb1a53ef91dc21027" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 34213 } } peers { permanent_uuid: "096a81e294374bb9aec1df09659f4d06" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34727 } } peers { permanent_uuid: "7264696b09d34a7580f9b3a89800d712" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 34691 } } }
I20260811 09:07:44.312287 32367 tablet_replica.cc:442] T dbb51c1bae6c408f9286965085478649 P 44fb00a4d0c3453bb1a53ef91dc21027: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 09:07:44.312726 32363 ts_tablet_manager.cc:1428] T dbb51c1bae6c408f9286965085478649 P 44fb00a4d0c3453bb1a53ef91dc21027: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260811 09:07:44.321887 32360 tablet_bootstrap.cc:492] T dbb51c1bae6c408f9286965085478649 P 7264696b09d34a7580f9b3a89800d712: No bootstrap required, opened a new log
I20260811 09:07:44.322113 32360 ts_tablet_manager.cc:1397] T dbb51c1bae6c408f9286965085478649 P 7264696b09d34a7580f9b3a89800d712: Time spent bootstrapping tablet: real 0.022s user 0.001s sys 0.009s
I20260811 09:07:44.322970 32360 raft_consensus.cc:348] T dbb51c1bae6c408f9286965085478649 P 7264696b09d34a7580f9b3a89800d712 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44fb00a4d0c3453bb1a53ef91dc21027" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 34213 } } peers { permanent_uuid: "096a81e294374bb9aec1df09659f4d06" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34727 } } peers { permanent_uuid: "7264696b09d34a7580f9b3a89800d712" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 34691 } }
I20260811 09:07:44.323248 32360 raft_consensus.cc:374] T dbb51c1bae6c408f9286965085478649 P 7264696b09d34a7580f9b3a89800d712 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:07:44.323272 32358 tablet_bootstrap.cc:492] T dbb51c1bae6c408f9286965085478649 P 096a81e294374bb9aec1df09659f4d06: No bootstrap required, opened a new log
I20260811 09:07:44.323333 32360 raft_consensus.cc:729] T dbb51c1bae6c408f9286965085478649 P 7264696b09d34a7580f9b3a89800d712 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7264696b09d34a7580f9b3a89800d712, State: Initialized, Role: FOLLOWER
I20260811 09:07:44.323436 32358 ts_tablet_manager.cc:1397] T dbb51c1bae6c408f9286965085478649 P 096a81e294374bb9aec1df09659f4d06: Time spent bootstrapping tablet: real 0.026s user 0.009s sys 0.002s
I20260811 09:07:44.323634 32360 consensus_queue.cc:260] T dbb51c1bae6c408f9286965085478649 P 7264696b09d34a7580f9b3a89800d712 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44fb00a4d0c3453bb1a53ef91dc21027" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 34213 } } peers { permanent_uuid: "096a81e294374bb9aec1df09659f4d06" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34727 } } peers { permanent_uuid: "7264696b09d34a7580f9b3a89800d712" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 34691 } }
I20260811 09:07:44.324154 32372 tablet_replica.cc:439] T dbb51c1bae6c408f9286965085478649 P 7264696b09d34a7580f9b3a89800d712: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44fb00a4d0c3453bb1a53ef91dc21027" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 34213 } } peers { permanent_uuid: "096a81e294374bb9aec1df09659f4d06" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34727 } } peers { permanent_uuid: "7264696b09d34a7580f9b3a89800d712" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 34691 } } }
I20260811 09:07:44.324506 32372 tablet_replica.cc:442] T dbb51c1bae6c408f9286965085478649 P 7264696b09d34a7580f9b3a89800d712: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 09:07:44.325356 32358 raft_consensus.cc:348] T dbb51c1bae6c408f9286965085478649 P 096a81e294374bb9aec1df09659f4d06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44fb00a4d0c3453bb1a53ef91dc21027" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 34213 } } peers { permanent_uuid: "096a81e294374bb9aec1df09659f4d06" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34727 } } peers { permanent_uuid: "7264696b09d34a7580f9b3a89800d712" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 34691 } }
I20260811 09:07:44.325640 32358 raft_consensus.cc:374] T dbb51c1bae6c408f9286965085478649 P 096a81e294374bb9aec1df09659f4d06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:07:44.325734 32358 raft_consensus.cc:729] T dbb51c1bae6c408f9286965085478649 P 096a81e294374bb9aec1df09659f4d06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 096a81e294374bb9aec1df09659f4d06, State: Initialized, Role: FOLLOWER
I20260811 09:07:44.325817 32360 ts_tablet_manager.cc:1428] T dbb51c1bae6c408f9286965085478649 P 7264696b09d34a7580f9b3a89800d712: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s
I20260811 09:07:44.325989 32358 consensus_queue.cc:260] T dbb51c1bae6c408f9286965085478649 P 096a81e294374bb9aec1df09659f4d06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44fb00a4d0c3453bb1a53ef91dc21027" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 34213 } } peers { permanent_uuid: "096a81e294374bb9aec1df09659f4d06" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34727 } } peers { permanent_uuid: "7264696b09d34a7580f9b3a89800d712" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 34691 } }
I20260811 09:07:44.326414 32369 tablet_replica.cc:439] T dbb51c1bae6c408f9286965085478649 P 096a81e294374bb9aec1df09659f4d06: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44fb00a4d0c3453bb1a53ef91dc21027" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 34213 } } peers { permanent_uuid: "096a81e294374bb9aec1df09659f4d06" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34727 } } peers { permanent_uuid: "7264696b09d34a7580f9b3a89800d712" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 34691 } } }
I20260811 09:07:44.326737 32369 tablet_replica.cc:442] T dbb51c1bae6c408f9286965085478649 P 096a81e294374bb9aec1df09659f4d06: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 09:07:44.327108 32358 ts_tablet_manager.cc:1428] T dbb51c1bae6c408f9286965085478649 P 096a81e294374bb9aec1df09659f4d06: Time spent starting tablet: real 0.004s user 0.002s sys 0.001s
I20260811 09:07:44.341652 32367 raft_consensus.cc:482] T dbb51c1bae6c408f9286965085478649 P 44fb00a4d0c3453bb1a53ef91dc21027 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:07:44.341861 32367 raft_consensus.cc:504] T dbb51c1bae6c408f9286965085478649 P 44fb00a4d0c3453bb1a53ef91dc21027 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44fb00a4d0c3453bb1a53ef91dc21027" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 34213 } } peers { permanent_uuid: "096a81e294374bb9aec1df09659f4d06" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34727 } } peers { permanent_uuid: "7264696b09d34a7580f9b3a89800d712" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 34691 } }
I20260811 09:07:44.342545 32367 leader_election.cc:288] T dbb51c1bae6c408f9286965085478649 P 44fb00a4d0c3453bb1a53ef91dc21027 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 096a81e294374bb9aec1df09659f4d06 (127.28.154.130:34727), 7264696b09d34a7580f9b3a89800d712 (127.28.154.131:34691)
I20260811 09:07:44.343098 31987 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dbb51c1bae6c408f9286965085478649" candidate_uuid: "44fb00a4d0c3453bb1a53ef91dc21027" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "096a81e294374bb9aec1df09659f4d06" is_pre_election: true
I20260811 09:07:44.343129 32227 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dbb51c1bae6c408f9286965085478649" candidate_uuid: "44fb00a4d0c3453bb1a53ef91dc21027" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7264696b09d34a7580f9b3a89800d712" is_pre_election: true
I20260811 09:07:44.343353 31987 raft_consensus.cc:2454] T dbb51c1bae6c408f9286965085478649 P 096a81e294374bb9aec1df09659f4d06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 44fb00a4d0c3453bb1a53ef91dc21027 in term 0.
I20260811 09:07:44.343428 32227 raft_consensus.cc:2454] T dbb51c1bae6c408f9286965085478649 P 7264696b09d34a7580f9b3a89800d712 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 44fb00a4d0c3453bb1a53ef91dc21027 in term 0.
I20260811 09:07:44.343922 31544 leader_election.cc:302] T dbb51c1bae6c408f9286965085478649 P 44fb00a4d0c3453bb1a53ef91dc21027 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 096a81e294374bb9aec1df09659f4d06, 44fb00a4d0c3453bb1a53ef91dc21027; no voters:
I20260811 09:07:44.344233 32367 raft_consensus.cc:2781] T dbb51c1bae6c408f9286965085478649 P 44fb00a4d0c3453bb1a53ef91dc21027 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 09:07:44.344410 32367 raft_consensus.cc:482] T dbb51c1bae6c408f9286965085478649 P 44fb00a4d0c3453bb1a53ef91dc21027 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 09:07:44.344514 32367 raft_consensus.cc:3037] T dbb51c1bae6c408f9286965085478649 P 44fb00a4d0c3453bb1a53ef91dc21027 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:07:44.347796 32367 raft_consensus.cc:504] T dbb51c1bae6c408f9286965085478649 P 44fb00a4d0c3453bb1a53ef91dc21027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44fb00a4d0c3453bb1a53ef91dc21027" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 34213 } } peers { permanent_uuid: "096a81e294374bb9aec1df09659f4d06" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34727 } } peers { permanent_uuid: "7264696b09d34a7580f9b3a89800d712" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 34691 } }
I20260811 09:07:44.348618 32367 leader_election.cc:288] T dbb51c1bae6c408f9286965085478649 P 44fb00a4d0c3453bb1a53ef91dc21027 [CANDIDATE]: Term 1 election: Requested vote from peers 096a81e294374bb9aec1df09659f4d06 (127.28.154.130:34727), 7264696b09d34a7580f9b3a89800d712 (127.28.154.131:34691)
I20260811 09:07:44.349184 31987 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dbb51c1bae6c408f9286965085478649" candidate_uuid: "44fb00a4d0c3453bb1a53ef91dc21027" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "096a81e294374bb9aec1df09659f4d06"
I20260811 09:07:44.349231 32227 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dbb51c1bae6c408f9286965085478649" candidate_uuid: "44fb00a4d0c3453bb1a53ef91dc21027" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7264696b09d34a7580f9b3a89800d712"
I20260811 09:07:44.349466 32227 raft_consensus.cc:3037] T dbb51c1bae6c408f9286965085478649 P 7264696b09d34a7580f9b3a89800d712 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:07:44.349466 31987 raft_consensus.cc:3037] T dbb51c1bae6c408f9286965085478649 P 096a81e294374bb9aec1df09659f4d06 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:07:44.352434 32227 raft_consensus.cc:2454] T dbb51c1bae6c408f9286965085478649 P 7264696b09d34a7580f9b3a89800d712 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 44fb00a4d0c3453bb1a53ef91dc21027 in term 1.
I20260811 09:07:44.352429 31987 raft_consensus.cc:2454] T dbb51c1bae6c408f9286965085478649 P 096a81e294374bb9aec1df09659f4d06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 44fb00a4d0c3453bb1a53ef91dc21027 in term 1.
I20260811 09:07:44.352929 31550 leader_election.cc:302] T dbb51c1bae6c408f9286965085478649 P 44fb00a4d0c3453bb1a53ef91dc21027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 096a81e294374bb9aec1df09659f4d06, 44fb00a4d0c3453bb1a53ef91dc21027, 7264696b09d34a7580f9b3a89800d712; no voters:
I20260811 09:07:44.353305 32367 raft_consensus.cc:2781] T dbb51c1bae6c408f9286965085478649 P 44fb00a4d0c3453bb1a53ef91dc21027 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:07:44.353475 32367 raft_consensus.cc:686] T dbb51c1bae6c408f9286965085478649 P 44fb00a4d0c3453bb1a53ef91dc21027 [term 1 LEADER]: Becoming Leader. State: Replica: 44fb00a4d0c3453bb1a53ef91dc21027, State: Running, Role: LEADER
I20260811 09:07:44.353771 32367 consensus_queue.cc:237] T dbb51c1bae6c408f9286965085478649 P 44fb00a4d0c3453bb1a53ef91dc21027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44fb00a4d0c3453bb1a53ef91dc21027" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 34213 } } peers { permanent_uuid: "096a81e294374bb9aec1df09659f4d06" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34727 } } peers { permanent_uuid: "7264696b09d34a7580f9b3a89800d712" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 34691 } }
I20260811 09:07:44.356630 32377 tablet_replica.cc:439] T dbb51c1bae6c408f9286965085478649 P 44fb00a4d0c3453bb1a53ef91dc21027: TxnStatusTablet state changed. Reason: New leader 44fb00a4d0c3453bb1a53ef91dc21027. Latest consensus state: current_term: 1 leader_uuid: "44fb00a4d0c3453bb1a53ef91dc21027" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44fb00a4d0c3453bb1a53ef91dc21027" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 34213 } } peers { permanent_uuid: "096a81e294374bb9aec1df09659f4d06" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34727 } } peers { permanent_uuid: "7264696b09d34a7580f9b3a89800d712" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 34691 } } }
I20260811 09:07:44.356930 32377 tablet_replica.cc:442] T dbb51c1bae6c408f9286965085478649 P 44fb00a4d0c3453bb1a53ef91dc21027: This TxnStatusTablet replica's current role is: LEADER
I20260811 09:07:44.357596 31400 catalog_manager.cc:5495] T dbb51c1bae6c408f9286965085478649 P 44fb00a4d0c3453bb1a53ef91dc21027 reported cstate change: term changed from 0 to 1, leader changed from <none> to 44fb00a4d0c3453bb1a53ef91dc21027 (127.28.154.129). New cstate: current_term: 1 leader_uuid: "44fb00a4d0c3453bb1a53ef91dc21027" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44fb00a4d0c3453bb1a53ef91dc21027" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 34213 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "096a81e294374bb9aec1df09659f4d06" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7264696b09d34a7580f9b3a89800d712" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 34691 } health_report { overall_health: UNKNOWN } } }
I20260811 09:07:44.358552 32436 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 09:07:44.358902 32436 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 09:07:44.378871 32427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:44.373697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.129:34213 (local address 127.0.0.1:44680)
0811 09:07:44.373916 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:44.373938 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 09:07:44.374097 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:07:44.374527 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:07:44.374532 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:07:44.374544 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 09:07:44.374724 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:44.374733 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:44.375903 (+ 1170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:44.375909 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:44.376994 (+ 1085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:44.377009 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:44.377718 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:44.377724 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:44.377870 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:44.377929 (+ 59us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 09:07:44.378329 (+ 400us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 09:07:44.378352 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 09:07:44.378420 (+ 68us) client_negotiation.cc:240] Negotiation successful
0811 09:07:44.378582 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260811 09:07:44.378890 32353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:44.373859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44680 (local address 127.28.154.129:34213)
0811 09:07:44.374086 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:44.374093 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:44.374123 (+ 30us) server_negotiation.cc:407] Connection header received
0811 09:07:44.374302 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:44.374306 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:44.374397 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:44.374482 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:07:44.374865 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:44.375766 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:44.377139 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:44.377575 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:44.377695 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:44.378105 (+ 410us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:07:44.378231 (+ 126us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:07:44.378313 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:44.378462 (+ 149us) server_negotiation.cc:299] Negotiation successful
0811 09:07:44.378553 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 09:07:44.383693 32227 raft_consensus.cc:1261] T dbb51c1bae6c408f9286965085478649 P 7264696b09d34a7580f9b3a89800d712 [term 1 FOLLOWER]: Refusing update from remote peer 44fb00a4d0c3453bb1a53ef91dc21027: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:07:44.383713 31987 raft_consensus.cc:1261] T dbb51c1bae6c408f9286965085478649 P 096a81e294374bb9aec1df09659f4d06 [term 1 FOLLOWER]: Refusing update from remote peer 44fb00a4d0c3453bb1a53ef91dc21027: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:07:44.384327 32367 consensus_queue.cc:1035] T dbb51c1bae6c408f9286965085478649 P 44fb00a4d0c3453bb1a53ef91dc21027 [LEADER]: Connected to new peer: Peer: permanent_uuid: "096a81e294374bb9aec1df09659f4d06" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:07:44.384729 32377 consensus_queue.cc:1035] T dbb51c1bae6c408f9286965085478649 P 44fb00a4d0c3453bb1a53ef91dc21027 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7264696b09d34a7580f9b3a89800d712" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 34691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:07:44.389395 32372 tablet_replica.cc:439] T dbb51c1bae6c408f9286965085478649 P 7264696b09d34a7580f9b3a89800d712: TxnStatusTablet state changed. Reason: New leader 44fb00a4d0c3453bb1a53ef91dc21027. Latest consensus state: current_term: 1 leader_uuid: "44fb00a4d0c3453bb1a53ef91dc21027" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44fb00a4d0c3453bb1a53ef91dc21027" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 34213 } } peers { permanent_uuid: "096a81e294374bb9aec1df09659f4d06" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34727 } } peers { permanent_uuid: "7264696b09d34a7580f9b3a89800d712" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 34691 } } }
I20260811 09:07:44.389734 32372 tablet_replica.cc:442] T dbb51c1bae6c408f9286965085478649 P 7264696b09d34a7580f9b3a89800d712: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 09:07:44.390884 32377 tablet_replica.cc:439] T dbb51c1bae6c408f9286965085478649 P 44fb00a4d0c3453bb1a53ef91dc21027: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "44fb00a4d0c3453bb1a53ef91dc21027" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44fb00a4d0c3453bb1a53ef91dc21027" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 34213 } } peers { permanent_uuid: "096a81e294374bb9aec1df09659f4d06" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34727 } } peers { permanent_uuid: "7264696b09d34a7580f9b3a89800d712" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 34691 } } }
I20260811 09:07:44.391139 32377 tablet_replica.cc:442] T dbb51c1bae6c408f9286965085478649 P 44fb00a4d0c3453bb1a53ef91dc21027: This TxnStatusTablet replica's current role is: LEADER
I20260811 09:07:44.396179 32372 tablet_replica.cc:439] T dbb51c1bae6c408f9286965085478649 P 7264696b09d34a7580f9b3a89800d712: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "44fb00a4d0c3453bb1a53ef91dc21027" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44fb00a4d0c3453bb1a53ef91dc21027" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 34213 } } peers { permanent_uuid: "096a81e294374bb9aec1df09659f4d06" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34727 } } peers { permanent_uuid: "7264696b09d34a7580f9b3a89800d712" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 34691 } } }
I20260811 09:07:44.396539 32372 tablet_replica.cc:442] T dbb51c1bae6c408f9286965085478649 P 7264696b09d34a7580f9b3a89800d712: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 09:07:44.398358 32369 tablet_replica.cc:439] T dbb51c1bae6c408f9286965085478649 P 096a81e294374bb9aec1df09659f4d06: TxnStatusTablet state changed. Reason: New leader 44fb00a4d0c3453bb1a53ef91dc21027. Latest consensus state: current_term: 1 leader_uuid: "44fb00a4d0c3453bb1a53ef91dc21027" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44fb00a4d0c3453bb1a53ef91dc21027" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 34213 } } peers { permanent_uuid: "096a81e294374bb9aec1df09659f4d06" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34727 } } peers { permanent_uuid: "7264696b09d34a7580f9b3a89800d712" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 34691 } } }
I20260811 09:07:44.398693 32369 tablet_replica.cc:442] T dbb51c1bae6c408f9286965085478649 P 096a81e294374bb9aec1df09659f4d06: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 09:07:44.399900 32377 tablet_replica.cc:439] T dbb51c1bae6c408f9286965085478649 P 44fb00a4d0c3453bb1a53ef91dc21027: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "44fb00a4d0c3453bb1a53ef91dc21027" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44fb00a4d0c3453bb1a53ef91dc21027" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 34213 } } peers { permanent_uuid: "096a81e294374bb9aec1df09659f4d06" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34727 } } peers { permanent_uuid: "7264696b09d34a7580f9b3a89800d712" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 34691 } } }
I20260811 09:07:44.400198 32377 tablet_replica.cc:442] T dbb51c1bae6c408f9286965085478649 P 44fb00a4d0c3453bb1a53ef91dc21027: This TxnStatusTablet replica's current role is: LEADER
I20260811 09:07:44.403811 32369 tablet_replica.cc:439] T dbb51c1bae6c408f9286965085478649 P 096a81e294374bb9aec1df09659f4d06: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "44fb00a4d0c3453bb1a53ef91dc21027" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "44fb00a4d0c3453bb1a53ef91dc21027" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 34213 } } peers { permanent_uuid: "096a81e294374bb9aec1df09659f4d06" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34727 } } peers { permanent_uuid: "7264696b09d34a7580f9b3a89800d712" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 34691 } } }
I20260811 09:07:44.404053 32369 tablet_replica.cc:442] T dbb51c1bae6c408f9286965085478649 P 096a81e294374bb9aec1df09659f4d06: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 09:07:44.432001 32353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:44.418572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.129:60769 (local address 127.28.154.129:34213)
0811 09:07:44.418751 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:44.418758 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:44.424925 (+ 6167us) server_negotiation.cc:407] Connection header received
0811 09:07:44.425185 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:44.425191 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:44.425279 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:44.425382 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:07:44.426113 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:44.427285 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:44.429732 (+ 2447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:44.430410 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:44.430534 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:44.431118 (+ 584us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:07:44.431290 (+ 172us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:07:44.431372 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:44.431512 (+ 140us) server_negotiation.cc:299] Negotiation successful
0811 09:07:44.431636 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":23,"server-negotiator.queue_time_us":63}
I20260811 09:07:44.432163 32448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:44.418464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.129:34213 (local address 127.28.154.129:60769)
0811 09:07:44.424777 (+ 6313us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:44.424801 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 09:07:44.425012 (+ 211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:07:44.425431 (+ 419us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:07:44.425436 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:07:44.425449 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 09:07:44.425862 (+ 413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:44.425873 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:44.427438 (+ 1565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:44.427443 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:44.429565 (+ 2122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:44.429583 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:44.430632 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:44.430640 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:44.430842 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:44.430928 (+ 86us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 09:07:44.431394 (+ 466us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 09:07:44.431406 (+ 12us) client_negotiation.cc:769] Sending connection context
0811 09:07:44.431486 (+ 80us) client_negotiation.cc:240] Negotiation successful
0811 09:07:44.431689 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6086,"mutex_wait_us":1,"thread_start_us":5934,"threads_started":1}
I20260811 09:07:44.444880 32227 raft_consensus.cc:1261] T 4f59ca5dde514bf88072bb7d7aa3aa42 P 7264696b09d34a7580f9b3a89800d712 [term 1 FOLLOWER]: Refusing update from remote peer 44fb00a4d0c3453bb1a53ef91dc21027: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:07:44.444881 31987 raft_consensus.cc:1261] T 4f59ca5dde514bf88072bb7d7aa3aa42 P 096a81e294374bb9aec1df09659f4d06 [term 1 FOLLOWER]: Refusing update from remote peer 44fb00a4d0c3453bb1a53ef91dc21027: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:07:44.445490 32367 consensus_queue.cc:1035] T 4f59ca5dde514bf88072bb7d7aa3aa42 P 44fb00a4d0c3453bb1a53ef91dc21027 [LEADER]: Connected to new peer: Peer: permanent_uuid: "096a81e294374bb9aec1df09659f4d06" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:07:44.446156 32377 consensus_queue.cc:1035] T 4f59ca5dde514bf88072bb7d7aa3aa42 P 44fb00a4d0c3453bb1a53ef91dc21027 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7264696b09d34a7580f9b3a89800d712" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 34691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:07:44.452365 32384 mvcc.cc:205] Tried to move back new op lower bound from 7317255227160973312 to 7317255226178392064. Current Snapshot: MvccSnapshot[applied={T|T < 7317255227160973312}]
I20260811 09:07:44.515554 31400 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38224:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:07:44.515805 31400 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38224:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:07:44.518941 31400 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f6cfaebcd30e476f93f1c0f1154871c7: Sending DeleteTablet for 3 replicas of tablet 4f59ca5dde514bf88072bb7d7aa3aa42
I20260811 09:07:44.520031 31940 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f59ca5dde514bf88072bb7d7aa3aa42 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:07:44 UTC) from {username='kudu', principal='kudu/127.28.154.190@KRBTEST.COM'} at 127.0.0.1:47398
I20260811 09:07:44.520193 31659 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f59ca5dde514bf88072bb7d7aa3aa42 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:07:44 UTC) from {username='kudu', principal='kudu/127.28.154.190@KRBTEST.COM'} at 127.0.0.1:44664
I20260811 09:07:44.520323 32407 tablet_replica.cc:330] T 4f59ca5dde514bf88072bb7d7aa3aa42 P 096a81e294374bb9aec1df09659f4d06: stopping tablet replica
I20260811 09:07:44.520377 32187 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f59ca5dde514bf88072bb7d7aa3aa42 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:07:44 UTC) from {username='kudu', principal='kudu/127.28.154.190@KRBTEST.COM'} at 127.0.0.1:60996
I20260811 09:07:44.520591 32405 tablet_replica.cc:330] T 4f59ca5dde514bf88072bb7d7aa3aa42 P 44fb00a4d0c3453bb1a53ef91dc21027: stopping tablet replica
I20260811 09:07:44.520637 32407 raft_consensus.cc:2229] T 4f59ca5dde514bf88072bb7d7aa3aa42 P 096a81e294374bb9aec1df09659f4d06 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:07:44.520787 32405 raft_consensus.cc:2229] T 4f59ca5dde514bf88072bb7d7aa3aa42 P 44fb00a4d0c3453bb1a53ef91dc21027 [term 1 LEADER]: Raft consensus shutting down.
I20260811 09:07:44.520841 32407 raft_consensus.cc:2258] T 4f59ca5dde514bf88072bb7d7aa3aa42 P 096a81e294374bb9aec1df09659f4d06 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:07:44.520931 32460 tablet_replica.cc:330] T 4f59ca5dde514bf88072bb7d7aa3aa42 P 7264696b09d34a7580f9b3a89800d712: stopping tablet replica
I20260811 09:07:44.521107 32460 raft_consensus.cc:2229] T 4f59ca5dde514bf88072bb7d7aa3aa42 P 7264696b09d34a7580f9b3a89800d712 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:07:44.521123 32405 raft_consensus.cc:2258] T 4f59ca5dde514bf88072bb7d7aa3aa42 P 44fb00a4d0c3453bb1a53ef91dc21027 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:07:44.521720 32460 raft_consensus.cc:2258] T 4f59ca5dde514bf88072bb7d7aa3aa42 P 7264696b09d34a7580f9b3a89800d712 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:07:44.522423 32407 ts_tablet_manager.cc:1905] T 4f59ca5dde514bf88072bb7d7aa3aa42 P 096a81e294374bb9aec1df09659f4d06: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:07:44.523305 32460 ts_tablet_manager.cc:1905] T 4f59ca5dde514bf88072bb7d7aa3aa42 P 7264696b09d34a7580f9b3a89800d712: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:07:44.523370 32405 ts_tablet_manager.cc:1905] T 4f59ca5dde514bf88072bb7d7aa3aa42 P 44fb00a4d0c3453bb1a53ef91dc21027: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:07:44.529992 32407 ts_tablet_manager.cc:1918] T 4f59ca5dde514bf88072bb7d7aa3aa42 P 096a81e294374bb9aec1df09659f4d06: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 09:07:44.530242 32407 log.cc:1196] T 4f59ca5dde514bf88072bb7d7aa3aa42 P 096a81e294374bb9aec1df09659f4d06: Deleting WAL directory at /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439250192320-29290-0/minicluster-data/ts-1/wal/wals/4f59ca5dde514bf88072bb7d7aa3aa42
I20260811 09:07:44.530786 32407 ts_tablet_manager.cc:1939] T 4f59ca5dde514bf88072bb7d7aa3aa42 P 096a81e294374bb9aec1df09659f4d06: Deleting consensus metadata
I20260811 09:07:44.531736 32460 ts_tablet_manager.cc:1918] T 4f59ca5dde514bf88072bb7d7aa3aa42 P 7264696b09d34a7580f9b3a89800d712: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 09:07:44.531885 32460 log.cc:1196] T 4f59ca5dde514bf88072bb7d7aa3aa42 P 7264696b09d34a7580f9b3a89800d712: Deleting WAL directory at /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439250192320-29290-0/minicluster-data/ts-2/wal/wals/4f59ca5dde514bf88072bb7d7aa3aa42
I20260811 09:07:44.532203 31379 catalog_manager.cc:4841] TS 096a81e294374bb9aec1df09659f4d06 (127.28.154.130:34727): tablet 4f59ca5dde514bf88072bb7d7aa3aa42 (table test-table [id=747620bab7b74dbabdb7f8c61012397a]) successfully deleted
I20260811 09:07:44.532331 32460 ts_tablet_manager.cc:1939] T 4f59ca5dde514bf88072bb7d7aa3aa42 P 7264696b09d34a7580f9b3a89800d712: Deleting consensus metadata
I20260811 09:07:44.533473 32405 ts_tablet_manager.cc:1918] T 4f59ca5dde514bf88072bb7d7aa3aa42 P 44fb00a4d0c3453bb1a53ef91dc21027: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 09:07:44.533458 31386 catalog_manager.cc:4841] TS 7264696b09d34a7580f9b3a89800d712 (127.28.154.131:34691): tablet 4f59ca5dde514bf88072bb7d7aa3aa42 (table test-table [id=747620bab7b74dbabdb7f8c61012397a]) successfully deleted
I20260811 09:07:44.533699 32405 log.cc:1196] T 4f59ca5dde514bf88072bb7d7aa3aa42 P 44fb00a4d0c3453bb1a53ef91dc21027: Deleting WAL directory at /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786439250192320-29290-0/minicluster-data/ts-0/wal/wals/4f59ca5dde514bf88072bb7d7aa3aa42
I20260811 09:07:44.534214 32405 ts_tablet_manager.cc:1939] T 4f59ca5dde514bf88072bb7d7aa3aa42 P 44fb00a4d0c3453bb1a53ef91dc21027: Deleting consensus metadata
I20260811 09:07:44.535643 31383 catalog_manager.cc:4841] TS 44fb00a4d0c3453bb1a53ef91dc21027 (127.28.154.129:34213): tablet 4f59ca5dde514bf88072bb7d7aa3aa42 (table test-table [id=747620bab7b74dbabdb7f8c61012397a]) successfully deleted
Aug 11 09:07:44 dist-test-slave-xt4k krb5kdc[31315](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439263, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.28.154.129@KRBTEST.COM
I20260811 09:07:44.556782 32353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:44.537598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44690 (local address 127.28.154.129:34213)
0811 09:07:44.537829 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:44.537836 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:44.538289 (+ 453us) server_negotiation.cc:407] Connection header received
0811 09:07:44.538521 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:44.538525 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:44.538596 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:44.538695 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:44.540677 (+ 1982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:44.541507 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:44.543616 (+ 2109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:44.544463 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:44.544586 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:44.550408 (+ 5822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:07:44.550433 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:07:44.550454 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:07:44.550500 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:07:44.553720 (+ 3220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:44.554484 (+ 764us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:44.554491 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:44.554498 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:44.554653 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:44.555050 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:44.555054 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:44.555057 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:44.555477 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:07:44.555794 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:44.556167 (+ 373us) server_negotiation.cc:299] Negotiation successful
0811 09:07:44.556450 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
W20260811 09:07:44.557590 31812 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44690
I20260811 09:07:44.575433 32428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:44.560536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38250 (local address 127.28.154.190:45807)
0811 09:07:44.560699 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:44.560703 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:44.560863 (+ 160us) server_negotiation.cc:407] Connection header received
0811 09:07:44.561027 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:44.561052 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:44.561142 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:44.561259 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:44.562565 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:44.563719 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:44.565321 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:44.566059 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:44.566198 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:44.568708 (+ 2510us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:07:44.568716 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:07:44.568721 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:07:44.568771 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:07:44.572325 (+ 3554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:44.573129 (+ 804us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:44.573135 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:44.573139 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:44.573298 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:44.573770 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:44.573776 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:44.573780 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:44.574191 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:07:44.574473 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:44.574831 (+ 358us) server_negotiation.cc:299] Negotiation successful
0811 09:07:44.575077 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
W20260811 09:07:44.576125 31400 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38250
I20260811 09:07:44.578505 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 31517
I20260811 09:07:44.592098 32353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:44.579472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44704 (local address 127.28.154.129:34213)
0811 09:07:44.579664 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:44.579670 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:44.580026 (+ 356us) server_negotiation.cc:407] Connection header received
0811 09:07:44.580168 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:44.580174 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:44.580263 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:44.580372 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:44.581788 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:44.582554 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:44.583841 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:44.584401 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:44.584525 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:44.586789 (+ 2264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:07:44.586794 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:07:44.586797 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:07:44.586832 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:07:44.589537 (+ 2705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:44.590156 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:44.590162 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:44.590166 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:44.590309 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:44.590679 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:44.590685 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:44.590689 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:44.590958 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:07:44.591252 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:44.591508 (+ 256us) server_negotiation.cc:299] Negotiation successful
0811 09:07:44.591740 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
W20260811 09:07:44.592794 31812 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44704
W20260811 09:07:44.593314 29290 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 31517: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 09:07:44.593474 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 31517
I20260811 09:07:44.636397 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 31825
Aug 11 09:07:44 dist-test-slave-xt4k krb5kdc[31315](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439263, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.28.154.130@KRBTEST.COM
I20260811 09:07:44.654229 32468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:44.637331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47408 (local address 127.28.154.130:34727)
0811 09:07:44.637906 (+ 575us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:44.637913 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:44.637932 (+ 19us) server_negotiation.cc:407] Connection header received
0811 09:07:44.637994 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:44.638008 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:44.638086 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:44.638202 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:44.639467 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:44.640760 (+ 1293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:44.642138 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:44.642817 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:44.642940 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:44.647468 (+ 4528us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:07:44.647486 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:07:44.647505 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:07:44.647555 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:07:44.651096 (+ 3541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:44.651836 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:44.651845 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:44.651847 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:44.652025 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:44.652558 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:44.652563 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:44.652565 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:44.653101 (+ 536us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:07:44.653372 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:44.653615 (+ 243us) server_negotiation.cc:299] Negotiation successful
0811 09:07:44.653909 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":433,"thread_start_us":284,"threads_started":1}
W20260811 09:07:44.655118 32057 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47408
W20260811 09:07:44.655925 29290 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 31825: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 09:07:44.656127 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 31825
I20260811 09:07:44.675958 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 32072
Aug 11 09:07:44 dist-test-slave-xt4k krb5kdc[31315](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439263, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.28.154.131@KRBTEST.COM
I20260811 09:07:44.693046 32470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:44.676818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32772 (local address 127.28.154.131:34691)
0811 09:07:44.677393 (+ 575us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:44.677398 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:44.677418 (+ 20us) server_negotiation.cc:407] Connection header received
0811 09:07:44.677474 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:44.677478 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:44.677550 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:44.677665 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:44.679349 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:44.680217 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:44.681795 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:44.682319 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:44.682449 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:44.686966 (+ 4517us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:07:44.686984 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:07:44.687003 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:07:44.687048 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:07:44.690385 (+ 3337us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:44.691146 (+ 761us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:44.691154 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:44.691157 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:44.691316 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:44.691644 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:44.691647 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:44.691649 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:44.692006 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:07:44.692254 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:44.692507 (+ 253us) server_negotiation.cc:299] Negotiation successful
0811 09:07:44.692701 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":461,"thread_start_us":332,"threads_started":1}
W20260811 09:07:44.693815 32314 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:32772
W20260811 09:07:44.694444 29290 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 32072: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 09:07:44.694588 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 32072
I20260811 09:07:44.728296 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 31340
I20260811 09:07:44.729502 31468 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:07:44.869063 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 31340
2026-08-11T09:07:44Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (5182 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439250192320-29290-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:07:44 dist-test-slave-xt4k krb5kdc[32478](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:07:44 dist-test-slave-xt4k krb5kdc[32478](info): set up 2 sockets
Aug 11 09:07:44 dist-test-slave-xt4k krb5kdc[32478](info): commencing operation
krb5kdc: starting...
W20260811 09:07:46.132230 29290 mini_kdc.cc:121] Time spent starting KDC: real 1.213s user 0.001s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439250192320-29290-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439250192320-29290-0/krb5kdc/test-user.keytab.
Aug 11 09:07:46 dist-test-slave-xt4k krb5kdc[32478](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439266, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:07:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:07:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.154.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.154.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.154.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab.
Entry for principal kudu/127.28.154.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab.
I20260811 09:07:46.353006 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439250192320-29290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439250192320-29290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439250192320-29290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.154.190:35229
--webserver_interface=127.28.154.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab
--principal=kudu/127.28.154.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:38531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.154.190:35229
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:07:46.685079 32496 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:07:46.685413 32496 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:07:46.685523 32496 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:07:46.693899 32496 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:07:46.694041 32496 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 09:07:46.694095 32496 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:07:46.694188 32496 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:07:46.694253 32496 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:07:46.694347 32496 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:07:46.705228 32496 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:38531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439250192320-29290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439250192320-29290-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.154.190:35229
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab
--principal=kudu/127.28.154.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.154.190:35229
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.154.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439250192320-29290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:07:46.706825 32496 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:07:46.709232 32496 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:07:46.721531 32514 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:07:46.721659 32504 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:07:46.721632 32506 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:07:46.722548 32496 server_base.cc:1034] running on GCE node
I20260811 09:07:46.723341 32496 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:07:46.724790 32496 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:07:46.725994 32496 hybrid_clock.cc:648] HybridClock initialized: now 1786439266725937 us; error 35 us; skew 500 ppm
I20260811 09:07:46.726363 32496 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:07:46 dist-test-slave-xt4k krb5kdc[32478](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439266, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:07:46.729774 32496 init.cc:377] Logged in from keytab as kudu/127.28.154.190@KRBTEST.COM (short username kudu)
I20260811 09:07:46.731307 32496 webserver.cc:460] Webserver started at http://127.28.154.190:40127/ using document root <none> and password file <none>
I20260811 09:07:46.732030 32496 fs_manager.cc:359] Metadata directory not provided
I20260811 09:07:46.732129 32496 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:07:46.732474 32496 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:07:46.735463 32496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439250192320-29290-0/minicluster-data/master-0/data/instance:
uuid: "49c59055f30d4f5e8fc4a7bfebbdd512"
format_stamp: "Formatted at 2026-08-11 09:07:46 on dist-test-slave-xt4k"
server_key: "97b5f4ad2bd5f1f1b4e45fb9bb763f14"
server_key_iv: "85fadacfb7e58bb6a0896d04db0ce485"
server_key_version: "encryptionkey@0"
I20260811 09:07:46.736227 32496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439250192320-29290-0/minicluster-data/master-0/wal/instance:
uuid: "49c59055f30d4f5e8fc4a7bfebbdd512"
format_stamp: "Formatted at 2026-08-11 09:07:46 on dist-test-slave-xt4k"
server_key: "97b5f4ad2bd5f1f1b4e45fb9bb763f14"
server_key_iv: "85fadacfb7e58bb6a0896d04db0ce485"
server_key_version: "encryptionkey@0"
I20260811 09:07:46.741475 32496 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 09:07:46.744911 32532 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:07:46.746230 32496 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 09:07:46.746380 32496 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439250192320-29290-0/minicluster-data/master-0/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439250192320-29290-0/minicluster-data/master-0/wal
uuid: "49c59055f30d4f5e8fc4a7bfebbdd512"
format_stamp: "Formatted at 2026-08-11 09:07:46 on dist-test-slave-xt4k"
server_key: "97b5f4ad2bd5f1f1b4e45fb9bb763f14"
server_key_iv: "85fadacfb7e58bb6a0896d04db0ce485"
server_key_version: "encryptionkey@0"
I20260811 09:07:46.746573 32496 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439250192320-29290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439250192320-29290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439250192320-29290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:07:46.759156 32496 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:07:46.763125 32496 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:07:46.763347 32496 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:07:46.783704 32496 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.190:35229
I20260811 09:07:46.783730 32664 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.190:35229 every 8 connection(s)
I20260811 09:07:46.785101 32496 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
I20260811 09:07:46.789083 32666 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:07:46.793830 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 32496
I20260811 09:07:46.794008 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439250192320-29290-0/minicluster-data/master-0/wal/instance
I20260811 09:07:46.794642 29290 external_mini_cluster.cc:1428] Setting key bd9fde8701ffdbdb9ece7593915c153e
I20260811 09:07:46.801932 32666 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49c59055f30d4f5e8fc4a7bfebbdd512: Bootstrap starting.
I20260811 09:07:46.805660 32666 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 49c59055f30d4f5e8fc4a7bfebbdd512: Neither blocks nor log segments found. Creating new log.
I20260811 09:07:46.806849 32666 log.cc:824] T 00000000000000000000000000000000 P 49c59055f30d4f5e8fc4a7bfebbdd512: Log is configured to *not* fsync() on all Append() calls
Aug 11 09:07:46 dist-test-slave-xt4k krb5kdc[32478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439266, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.154.190@KRBTEST.COM
I20260811 09:07:46.810250 32666 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49c59055f30d4f5e8fc4a7bfebbdd512: No bootstrap required, opened a new log
I20260811 09:07:46.818538 32666 raft_consensus.cc:348] T 00000000000000000000000000000000 P 49c59055f30d4f5e8fc4a7bfebbdd512 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49c59055f30d4f5e8fc4a7bfebbdd512" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 35229 } }
I20260811 09:07:46.818955 32666 raft_consensus.cc:374] T 00000000000000000000000000000000 P 49c59055f30d4f5e8fc4a7bfebbdd512 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:07:46.819059 32666 raft_consensus.cc:729] T 00000000000000000000000000000000 P 49c59055f30d4f5e8fc4a7bfebbdd512 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49c59055f30d4f5e8fc4a7bfebbdd512, State: Initialized, Role: FOLLOWER
I20260811 09:07:46.819337 32669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:46.796548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47726 (local address 127.28.154.190:35229)
0811 09:07:46.797519 (+ 971us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:46.797533 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:46.797592 (+ 59us) server_negotiation.cc:407] Connection header received
0811 09:07:46.800000 (+ 2408us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:46.800033 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:46.800597 (+ 564us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:46.801014 (+ 417us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:46.802800 (+ 1786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:46.804113 (+ 1313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:46.805910 (+ 1797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:46.806610 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:46.806755 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:46.811218 (+ 4463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:07:46.811226 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:07:46.811243 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:07:46.811283 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:07:46.814907 (+ 3624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:46.815658 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:46.815668 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:46.815675 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:46.815867 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:46.816260 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:46.816263 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:46.816265 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:46.816947 (+ 682us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:07:46.817290 (+ 343us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:46.817743 (+ 453us) server_negotiation.cc:299] Negotiation successful
0811 09:07:46.818172 (+ 429us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":709,"thread_start_us":440,"threads_started":1}
I20260811 09:07:46.819720 32666 consensus_queue.cc:260] T 00000000000000000000000000000000 P 49c59055f30d4f5e8fc4a7bfebbdd512 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49c59055f30d4f5e8fc4a7bfebbdd512" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 35229 } }
I20260811 09:07:46.819980 32666 raft_consensus.cc:388] T 00000000000000000000000000000000 P 49c59055f30d4f5e8fc4a7bfebbdd512 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:07:46.820115 32666 raft_consensus.cc:482] T 00000000000000000000000000000000 P 49c59055f30d4f5e8fc4a7bfebbdd512 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:07:46.820281 32666 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 49c59055f30d4f5e8fc4a7bfebbdd512 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:07:46.823423 32666 raft_consensus.cc:504] T 00000000000000000000000000000000 P 49c59055f30d4f5e8fc4a7bfebbdd512 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49c59055f30d4f5e8fc4a7bfebbdd512" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 35229 } }
I20260811 09:07:46.823935 32666 leader_election.cc:302] T 00000000000000000000000000000000 P 49c59055f30d4f5e8fc4a7bfebbdd512 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 49c59055f30d4f5e8fc4a7bfebbdd512; no voters:
I20260811 09:07:46.824543 32666 leader_election.cc:288] T 00000000000000000000000000000000 P 49c59055f30d4f5e8fc4a7bfebbdd512 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:07:46.824656 32675 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 49c59055f30d4f5e8fc4a7bfebbdd512 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:07:46.825309 32675 raft_consensus.cc:686] T 00000000000000000000000000000000 P 49c59055f30d4f5e8fc4a7bfebbdd512 [term 1 LEADER]: Becoming Leader. State: Replica: 49c59055f30d4f5e8fc4a7bfebbdd512, State: Running, Role: LEADER
I20260811 09:07:46.825879 32675 consensus_queue.cc:237] T 00000000000000000000000000000000 P 49c59055f30d4f5e8fc4a7bfebbdd512 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49c59055f30d4f5e8fc4a7bfebbdd512" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 35229 } }
I20260811 09:07:46.826622 32666 sys_catalog.cc:564] T 00000000000000000000000000000000 P 49c59055f30d4f5e8fc4a7bfebbdd512 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:07:46.828825 32680 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49c59055f30d4f5e8fc4a7bfebbdd512 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 49c59055f30d4f5e8fc4a7bfebbdd512. Latest consensus state: current_term: 1 leader_uuid: "49c59055f30d4f5e8fc4a7bfebbdd512" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49c59055f30d4f5e8fc4a7bfebbdd512" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 35229 } } }
I20260811 09:07:46.828801 32679 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49c59055f30d4f5e8fc4a7bfebbdd512 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "49c59055f30d4f5e8fc4a7bfebbdd512" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49c59055f30d4f5e8fc4a7bfebbdd512" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 35229 } } }
I20260811 09:07:46.829232 32679 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49c59055f30d4f5e8fc4a7bfebbdd512 [sys.catalog]: This master's current role is: LEADER
I20260811 09:07:46.831919 32680 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49c59055f30d4f5e8fc4a7bfebbdd512 [sys.catalog]: This master's current role is: LEADER
I20260811 09:07:46.832152 32683 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:07:46.837428 32683 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:07:46.847708 32683 catalog_manager.cc:1347] Generated new cluster ID: 08efe91ea0904d91ad625513682bc8e2
I20260811 09:07:46.847837 32683 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:07:46.893262 32683 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:07:46.895207 32683 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:07:46.907716 32683 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 49c59055f30d4f5e8fc4a7bfebbdd512: Generated new TSK 0
I20260811 09:07:46.908959 32683 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.28.154.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.154.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.154.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.129.keytab.
Entry for principal kudu/127.28.154.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.129.keytab.
I20260811 09:07:46.968317 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439250192320-29290-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439250192320-29290-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439250192320-29290-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439250192320-29290-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.154.129:0
--local_ip_for_outbound_sockets=127.28.154.129
--webserver_interface=127.28.154.129
--webserver_port=0
--tserver_master_addrs=127.28.154.190:35229
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.129.keytab
--principal=kudu/127.28.154.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:38531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:07:47.310276 32707 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:07:47.310609 32707 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:07:47.310779 32707 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:07:47.318892 32707 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:07:47.319064 32707 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:07:47.319202 32707 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 09:07:47.319273 32707 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 09:07:47.319381 32707 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.154.129
I20260811 09:07:47.330125 32707 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:38531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439250192320-29290-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439250192320-29290-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.129.keytab
--principal=kudu/127.28.154.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.154.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439250192320-29290-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.154.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.28.154.190:35229
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.154.129
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439250192320-29290-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:07:47.331552 32707 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:07:47.333146 32707 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:07:47.346812 32719 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:07:47.346843 32725 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:07:47.347025 32718 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:07:47.347602 32707 server_base.cc:1034] running on GCE node
I20260811 09:07:47.348291 32707 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:07:47.349428 32707 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:07:47.350735 32707 hybrid_clock.cc:648] HybridClock initialized: now 1786439267350618 us; error 58 us; skew 500 ppm
I20260811 09:07:47.351120 32707 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:07:47 dist-test-slave-xt4k krb5kdc[32478](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439267, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:07:47.358080 32707 init.cc:377] Logged in from keytab as kudu/127.28.154.129@KRBTEST.COM (short username kudu)
I20260811 09:07:47.359593 32707 webserver.cc:460] Webserver started at http://127.28.154.129:38585/ using document root <none> and password file <none>
I20260811 09:07:47.360375 32707 fs_manager.cc:359] Metadata directory not provided
I20260811 09:07:47.360494 32707 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:07:47.360781 32707 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:07:47.363732 32707 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439250192320-29290-0/minicluster-data/ts-0/data/instance:
uuid: "b66124a6ca6244748b38ebb07c6ca40e"
format_stamp: "Formatted at 2026-08-11 09:07:47 on dist-test-slave-xt4k"
server_key: "431cedda670886f0e8e02d97d7804fdb"
server_key_iv: "7e5b4b04de78df2760db252bb6c8494a"
server_key_version: "encryptionkey@0"
I20260811 09:07:47.364810 32707 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439250192320-29290-0/minicluster-data/ts-0/wal/instance:
uuid: "b66124a6ca6244748b38ebb07c6ca40e"
format_stamp: "Formatted at 2026-08-11 09:07:47 on dist-test-slave-xt4k"
server_key: "431cedda670886f0e8e02d97d7804fdb"
server_key_iv: "7e5b4b04de78df2760db252bb6c8494a"
server_key_version: "encryptionkey@0"
I20260811 09:07:47.370329 32707 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 09:07:47.374058 32736 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:07:47.375566 32707 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 09:07:47.375756 32707 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439250192320-29290-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439250192320-29290-0/minicluster-data/ts-0/wal
uuid: "b66124a6ca6244748b38ebb07c6ca40e"
format_stamp: "Formatted at 2026-08-11 09:07:47 on dist-test-slave-xt4k"
server_key: "431cedda670886f0e8e02d97d7804fdb"
server_key_iv: "7e5b4b04de78df2760db252bb6c8494a"
server_key_version: "encryptionkey@0"
I20260811 09:07:47.375964 32707 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439250192320-29290-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439250192320-29290-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439250192320-29290-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:07:47.399523 32707 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:07:47.404175 32707 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:07:47.404523 32707 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:07:47.407372 32707 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:07:47.407505 32707 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:07:47.407624 32707 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:07:47.407725 32707 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 09:07:47 dist-test-slave-xt4k krb5kdc[32478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439267, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.129@KRBTEST.COM for kudu/127.28.154.190@KRBTEST.COM
I20260811 09:07:47.430408 32755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:47.407948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.129:43371 (local address 127.28.154.190:35229)
0811 09:07:47.408686 (+ 738us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:47.408692 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:47.409719 (+ 1027us) server_negotiation.cc:407] Connection header received
0811 09:07:47.411806 (+ 2087us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:47.411812 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:47.411912 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:47.412045 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:47.414356 (+ 2311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:47.415548 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:47.417955 (+ 2407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:47.418483 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:47.418625 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:47.423494 (+ 4869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:07:47.423500 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:07:47.423504 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:07:47.423542 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:07:47.426646 (+ 3104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:47.427682 (+ 1036us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:47.427688 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:47.427692 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:47.427820 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:47.428503 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:47.428509 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:47.428513 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:47.428862 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:07:47.429009 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:47.429798 (+ 789us) server_negotiation.cc:299] Negotiation successful
0811 09:07:47.430043 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":614,"thread_start_us":492,"threads_started":1}
I20260811 09:07:47.431488 32757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:47.408665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:35229 (local address 127.28.154.129:43371)
0811 09:07:47.409526 (+ 861us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:47.409576 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 09:07:47.411516 (+ 1940us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:07:47.412323 (+ 807us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:07:47.412331 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:07:47.413086 (+ 755us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:07:47.414080 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:47.414108 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:47.415733 (+ 1625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:47.415742 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:47.417796 (+ 2054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:47.417813 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:47.418763 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:47.418770 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:47.419007 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:47.419814 (+ 807us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:07:47.419856 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:07:47.423167 (+ 3311us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:07:47.426809 (+ 3642us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:07:47.426820 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:07:47.426842 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:07:47.427548 (+ 706us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:07:47.427987 (+ 439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:07:47.427994 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:07:47.427998 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:07:47.428331 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:07:47.429004 (+ 673us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:07:47.429016 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:07:47.429411 (+ 395us) client_negotiation.cc:769] Sending connection context
0811 09:07:47.429672 (+ 261us) client_negotiation.cc:240] Negotiation successful
0811 09:07:47.430249 (+ 577us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":572,"thread_start_us":365,"threads_started":1}
I20260811 09:07:47.451982 32755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:47.439240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.129:55315 (local address 127.28.154.190:35229)
0811 09:07:47.439390 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:47.439394 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:47.439955 (+ 561us) server_negotiation.cc:407] Connection header received
0811 09:07:47.440135 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:47.440139 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:47.440204 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:47.440285 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:47.441585 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:47.442536 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:47.444283 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:47.444770 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:47.444884 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:47.446966 (+ 2082us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:07:47.446971 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:07:47.446974 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:07:47.447016 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:07:47.449395 (+ 2379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:47.450034 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:47.450039 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:47.450043 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:47.450176 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:47.450510 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:47.450515 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:47.450519 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:47.450817 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:07:47.450959 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:47.451387 (+ 428us) server_negotiation.cc:299] Negotiation successful
0811 09:07:47.451616 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 09:07:47.452127 438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:47.439286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:35229 (local address 127.28.154.129:55315)
0811 09:07:47.439855 (+ 569us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:47.439871 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 09:07:47.440017 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:07:47.440324 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:07:47.440328 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:07:47.440992 (+ 664us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:07:47.441441 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:47.441455 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:47.442670 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:47.442676 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:47.444146 (+ 1470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:47.444160 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:47.444963 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:47.444970 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:47.445125 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:47.445902 (+ 777us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:07:47.445919 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:07:47.446783 (+ 864us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:07:47.449520 (+ 2737us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:07:47.449526 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:07:47.449530 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:07:47.449927 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:07:47.450280 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:07:47.450285 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:07:47.450289 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:07:47.450410 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:07:47.450931 (+ 521us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:07:47.450936 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:07:47.451135 (+ 199us) client_negotiation.cc:769] Sending connection context
0811 09:07:47.451277 (+ 142us) client_negotiation.cc:240] Negotiation successful
0811 09:07:47.451646 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":308,"threads_started":1}
I20260811 09:07:47.459460 32707 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.129:41515
I20260811 09:07:47.459525 508 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.129:41515 every 8 connection(s)
I20260811 09:07:47.460777 32707 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439250192320-29290-0/minicluster-data/ts-0/data/info.pb
I20260811 09:07:47.462145 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 32707
I20260811 09:07:47.462337 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439250192320-29290-0/minicluster-data/ts-0/wal/instance
I20260811 09:07:47.463095 29290 external_mini_cluster.cc:1428] Setting key 6936c7f04d22acdac2ca07bdfdaa65f1
I20260811 09:07:47.462949 509 heartbeater.cc:347] Connected to a master server at 127.28.154.190:35229
I20260811 09:07:47.464180 509 heartbeater.cc:464] Registering TS with master...
I20260811 09:07:47.464937 509 heartbeater.cc:511] Master 127.28.154.190:35229 requested a full tablet report, sending...
I20260811 09:07:47.467031 32567 ts_manager.cc:194] Registered new tserver with Master: b66124a6ca6244748b38ebb07c6ca40e (127.28.154.129:41515)
I20260811 09:07:47.468582 32567 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.154.129@KRBTEST.COM'} at 127.28.154.129:43371
WARNING: no policy specified for kudu/127.28.154.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.154.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.154.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.130.keytab.
Entry for principal kudu/127.28.154.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.130.keytab.
I20260811 09:07:47.507918 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439250192320-29290-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439250192320-29290-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439250192320-29290-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439250192320-29290-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.154.130:0
--local_ip_for_outbound_sockets=127.28.154.130
--webserver_interface=127.28.154.130
--webserver_port=0
--tserver_master_addrs=127.28.154.190:35229
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.130.keytab
--principal=kudu/127.28.154.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:38531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:07:47.840248 517 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:07:47.840564 517 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:07:47.840674 517 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:07:47.847980 517 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:07:47.848098 517 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:07:47.848197 517 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 09:07:47.848249 517 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 09:07:47.848320 517 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.154.130
I20260811 09:07:47.858382 517 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:38531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439250192320-29290-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439250192320-29290-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.130.keytab
--principal=kudu/127.28.154.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.154.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439250192320-29290-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.154.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.28.154.190:35229
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.154.130
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439250192320-29290-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:07:47.859704 517 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:07:47.861081 517 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:07:47.874948 527 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:07:47.874926 530 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:07:47.874897 528 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:07:47.878026 517 server_base.cc:1034] running on GCE node
I20260811 09:07:47.878674 517 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:07:47.879724 517 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:07:47.880944 517 hybrid_clock.cc:648] HybridClock initialized: now 1786439267880868 us; error 63 us; skew 500 ppm
I20260811 09:07:47.881294 517 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:07:47 dist-test-slave-xt4k krb5kdc[32478](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439267, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:07:47.884680 517 init.cc:377] Logged in from keytab as kudu/127.28.154.130@KRBTEST.COM (short username kudu)
I20260811 09:07:47.886350 517 webserver.cc:460] Webserver started at http://127.28.154.130:41809/ using document root <none> and password file <none>
I20260811 09:07:47.887096 517 fs_manager.cc:359] Metadata directory not provided
I20260811 09:07:47.887207 517 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:07:47.887585 517 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:07:47.890695 517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439250192320-29290-0/minicluster-data/ts-1/data/instance:
uuid: "e8b65b15973c4d7d896a96567f97caee"
format_stamp: "Formatted at 2026-08-11 09:07:47 on dist-test-slave-xt4k"
server_key: "73b3e4cf3ae68056ff0b321c47c952f9"
server_key_iv: "0e65c6ef6f67cdf052695606abb358bf"
server_key_version: "encryptionkey@0"
I20260811 09:07:47.891484 517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439250192320-29290-0/minicluster-data/ts-1/wal/instance:
uuid: "e8b65b15973c4d7d896a96567f97caee"
format_stamp: "Formatted at 2026-08-11 09:07:47 on dist-test-slave-xt4k"
server_key: "73b3e4cf3ae68056ff0b321c47c952f9"
server_key_iv: "0e65c6ef6f67cdf052695606abb358bf"
server_key_version: "encryptionkey@0"
I20260811 09:07:47.896879 517 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 09:07:47.900638 538 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:07:47.901994 517 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 09:07:47.902164 517 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439250192320-29290-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439250192320-29290-0/minicluster-data/ts-1/wal
uuid: "e8b65b15973c4d7d896a96567f97caee"
format_stamp: "Formatted at 2026-08-11 09:07:47 on dist-test-slave-xt4k"
server_key: "73b3e4cf3ae68056ff0b321c47c952f9"
server_key_iv: "0e65c6ef6f67cdf052695606abb358bf"
server_key_version: "encryptionkey@0"
I20260811 09:07:47.902390 517 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439250192320-29290-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439250192320-29290-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439250192320-29290-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:07:47.928095 517 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:07:47.932381 517 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:07:47.932690 517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:07:47.935868 517 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:07:47.936058 517 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:07:47.936199 517 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:07:47.936288 517 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 09:07:47 dist-test-slave-xt4k krb5kdc[32478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439267, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.130@KRBTEST.COM for kudu/127.28.154.190@KRBTEST.COM
I20260811 09:07:47.958294 32755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:47.936108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.130:38153 (local address 127.28.154.190:35229)
0811 09:07:47.936316 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:47.936323 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:47.937782 (+ 1459us) server_negotiation.cc:407] Connection header received
0811 09:07:47.939847 (+ 2065us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:47.939853 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:47.939943 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:47.940045 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:47.942238 (+ 2193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:47.943177 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:47.946291 (+ 3114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:47.946788 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:47.946937 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:47.951911 (+ 4974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:07:47.951923 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:07:47.951927 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:07:47.951965 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:07:47.954592 (+ 2627us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:47.955429 (+ 837us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:47.955433 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:47.955436 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:47.955570 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:47.956249 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:47.956254 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:47.956257 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:47.956658 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:07:47.956840 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:47.957671 (+ 831us) server_negotiation.cc:299] Negotiation successful
0811 09:07:47.957935 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 09:07:47.959425 554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:47.936690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:35229 (local address 127.28.154.130:38153)
0811 09:07:47.937604 (+ 914us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:47.937652 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 09:07:47.939547 (+ 1895us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:07:47.940294 (+ 747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:07:47.940303 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:07:47.941094 (+ 791us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:07:47.942020 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:47.942060 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:47.943372 (+ 1312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:47.943379 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:47.946121 (+ 2742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:47.946139 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:47.947078 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:47.947087 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:47.947431 (+ 344us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:47.948288 (+ 857us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:07:47.948324 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:07:47.951651 (+ 3327us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:07:47.954765 (+ 3114us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:07:47.954777 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:07:47.954798 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:07:47.955282 (+ 484us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:07:47.955688 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:07:47.955694 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:07:47.955698 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:07:47.956057 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:07:47.956827 (+ 770us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:07:47.956835 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:07:47.957253 (+ 418us) client_negotiation.cc:769] Sending connection context
0811 09:07:47.957521 (+ 268us) client_negotiation.cc:240] Negotiation successful
0811 09:07:47.958101 (+ 580us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":607,"thread_start_us":338,"threads_started":1}
I20260811 09:07:47.981314 32755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:47.966541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.130:45175 (local address 127.28.154.190:35229)
0811 09:07:47.966749 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:47.966756 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:47.967360 (+ 604us) server_negotiation.cc:407] Connection header received
0811 09:07:47.967576 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:47.967579 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:47.967646 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:47.967725 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:47.969227 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:47.970043 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:47.971997 (+ 1954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:47.972492 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:47.972627 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:47.975251 (+ 2624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:07:47.975257 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:07:47.975262 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:07:47.975303 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:07:47.978508 (+ 3205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:47.979223 (+ 715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:47.979229 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:47.979232 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:47.979387 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:47.979737 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:47.979742 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:47.979745 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:47.980117 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:07:47.980256 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:47.980744 (+ 488us) server_negotiation.cc:299] Negotiation successful
0811 09:07:47.980956 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 09:07:47.981437 671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:47.966555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:35229 (local address 127.28.154.130:45175)
0811 09:07:47.967243 (+ 688us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:47.967266 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 09:07:47.967432 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:07:47.967771 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:07:47.967776 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:07:47.968498 (+ 722us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:07:47.969051 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:47.969076 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:47.970165 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:47.970171 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:47.971831 (+ 1660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:47.971848 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:47.972712 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:47.972720 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:47.972926 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:47.973675 (+ 749us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:07:47.973704 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:07:47.975021 (+ 1317us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:07:47.978662 (+ 3641us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:07:47.978668 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:07:47.978673 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:07:47.979090 (+ 417us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:07:47.979498 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:07:47.979503 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:07:47.979508 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:07:47.979636 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:07:47.980235 (+ 599us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:07:47.980239 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:07:47.980489 (+ 250us) client_negotiation.cc:769] Sending connection context
0811 09:07:47.980627 (+ 138us) client_negotiation.cc:240] Negotiation successful
0811 09:07:47.981001 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":520,"thread_start_us":350,"threads_started":1}
I20260811 09:07:47.993857 517 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.130:38009
I20260811 09:07:47.993930 759 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.130:38009 every 8 connection(s)
I20260811 09:07:47.995146 517 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439250192320-29290-0/minicluster-data/ts-1/data/info.pb
I20260811 09:07:47.997318 761 heartbeater.cc:347] Connected to a master server at 127.28.154.190:35229
I20260811 09:07:47.997529 761 heartbeater.cc:464] Registering TS with master...
I20260811 09:07:47.998203 761 heartbeater.cc:511] Master 127.28.154.190:35229 requested a full tablet report, sending...
I20260811 09:07:48.000020 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 517
I20260811 09:07:48.000041 32567 ts_manager.cc:194] Registered new tserver with Master: e8b65b15973c4d7d896a96567f97caee (127.28.154.130:38009)
I20260811 09:07:48.000186 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439250192320-29290-0/minicluster-data/ts-1/wal/instance
I20260811 09:07:48.000666 29290 external_mini_cluster.cc:1428] Setting key 5999cee510ccaa7cd52118366de378d3
I20260811 09:07:48.001212 32567 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.154.130@KRBTEST.COM'} at 127.28.154.130:38153
WARNING: no policy specified for kudu/127.28.154.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.154.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.154.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.131.keytab.
Entry for principal kudu/127.28.154.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.131.keytab.
I20260811 09:07:48.048183 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439250192320-29290-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439250192320-29290-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439250192320-29290-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439250192320-29290-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.154.131:0
--local_ip_for_outbound_sockets=127.28.154.131
--webserver_interface=127.28.154.131
--webserver_port=0
--tserver_master_addrs=127.28.154.190:35229
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.131.keytab
--principal=kudu/127.28.154.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:38531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:07:48.383843 768 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:07:48.384143 768 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:07:48.384248 768 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:07:48.391845 768 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:07:48.391973 768 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:07:48.392059 768 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 09:07:48.392112 768 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 09:07:48.392174 768 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.154.131
I20260811 09:07:48.402103 768 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:38531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439250192320-29290-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439250192320-29290-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.131.keytab
--principal=kudu/127.28.154.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.154.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439250192320-29290-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.154.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.28.154.190:35229
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.154.131
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439250192320-29290-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:07:48.403383 768 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:07:48.404737 768 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:07:48.418207 775 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:07:48.418458 777 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:07:48.418937 788 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:07:48.418973 768 server_base.cc:1034] running on GCE node
I20260811 09:07:48.419658 768 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:07:48.420644 768 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:07:48.421862 768 hybrid_clock.cc:648] HybridClock initialized: now 1786439268421811 us; error 43 us; skew 500 ppm
I20260811 09:07:48.422178 768 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:07:48 dist-test-slave-xt4k krb5kdc[32478](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439268, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:07:48.425416 768 init.cc:377] Logged in from keytab as kudu/127.28.154.131@KRBTEST.COM (short username kudu)
I20260811 09:07:48.427030 768 webserver.cc:460] Webserver started at http://127.28.154.131:40703/ using document root <none> and password file <none>
I20260811 09:07:48.427740 768 fs_manager.cc:359] Metadata directory not provided
I20260811 09:07:48.427860 768 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:07:48.428190 768 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:07:48.431273 768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439250192320-29290-0/minicluster-data/ts-2/data/instance:
uuid: "7136de9ce16f4703a928d1429952cdea"
format_stamp: "Formatted at 2026-08-11 09:07:48 on dist-test-slave-xt4k"
server_key: "c3f925243afbb9c08dc65e8655b9c8e5"
server_key_iv: "751b7e8cb53a66c8b31146db72c6d753"
server_key_version: "encryptionkey@0"
I20260811 09:07:48.432040 768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439250192320-29290-0/minicluster-data/ts-2/wal/instance:
uuid: "7136de9ce16f4703a928d1429952cdea"
format_stamp: "Formatted at 2026-08-11 09:07:48 on dist-test-slave-xt4k"
server_key: "c3f925243afbb9c08dc65e8655b9c8e5"
server_key_iv: "751b7e8cb53a66c8b31146db72c6d753"
server_key_version: "encryptionkey@0"
I20260811 09:07:48.437588 768 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260811 09:07:48.441525 801 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:07:48.442934 768 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 09:07:48.443095 768 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439250192320-29290-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439250192320-29290-0/minicluster-data/ts-2/wal
uuid: "7136de9ce16f4703a928d1429952cdea"
format_stamp: "Formatted at 2026-08-11 09:07:48 on dist-test-slave-xt4k"
server_key: "c3f925243afbb9c08dc65e8655b9c8e5"
server_key_iv: "751b7e8cb53a66c8b31146db72c6d753"
server_key_version: "encryptionkey@0"
I20260811 09:07:48.443293 768 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439250192320-29290-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439250192320-29290-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439250192320-29290-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:07:48.471941 509 heartbeater.cc:503] Master 127.28.154.190:35229 was elected leader, sending a full tablet report...
I20260811 09:07:48.472958 768 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:07:48.477501 768 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:07:48.477806 768 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:07:48.480793 768 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:07:48.480906 768 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:07:48.481019 768 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:07:48.481097 768 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 09:07:48 dist-test-slave-xt4k krb5kdc[32478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439268, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.131@KRBTEST.COM for kudu/127.28.154.190@KRBTEST.COM
I20260811 09:07:48.508666 32755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:48.481551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.131:55569 (local address 127.28.154.190:35229)
0811 09:07:48.481964 (+ 413us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:48.481970 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:48.483327 (+ 1357us) server_negotiation.cc:407] Connection header received
0811 09:07:48.486195 (+ 2868us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:48.486201 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:48.486299 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:48.486427 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:48.489085 (+ 2658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:48.490105 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:48.492496 (+ 2391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:48.493304 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:48.493456 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:48.500593 (+ 7137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:07:48.500602 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:07:48.500608 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:07:48.500662 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:07:48.504245 (+ 3583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:48.505309 (+ 1064us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:48.505316 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:48.505319 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:48.505498 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:48.506225 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:48.506232 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:48.506236 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:48.506714 (+ 478us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:07:48.506897 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:48.507984 (+ 1087us) server_negotiation.cc:299] Negotiation successful
0811 09:07:48.508249 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276}
I20260811 09:07:48.510139 820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:48.482198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:35229 (local address 127.28.154.131:55569)
0811 09:07:48.483133 (+ 935us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:48.483185 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 09:07:48.485819 (+ 2634us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:07:48.486742 (+ 923us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:07:48.486753 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:07:48.487734 (+ 981us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:07:48.488840 (+ 1106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:48.488891 (+ 51us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:48.490286 (+ 1395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:48.490290 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:48.492270 (+ 1980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:48.492294 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:48.493835 (+ 1541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:48.493847 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:48.494219 (+ 372us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:48.495071 (+ 852us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:07:48.495112 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:07:48.500119 (+ 5007us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:07:48.504432 (+ 4313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:07:48.504443 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:07:48.504464 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:07:48.505161 (+ 697us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:07:48.505649 (+ 488us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:07:48.505656 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:07:48.505661 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:07:48.506021 (+ 360us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:07:48.506904 (+ 883us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:07:48.506913 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:07:48.507529 (+ 616us) client_negotiation.cc:769] Sending connection context
0811 09:07:48.507848 (+ 319us) client_negotiation.cc:240] Negotiation successful
0811 09:07:48.508490 (+ 642us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":621,"thread_start_us":394,"threads_started":1}
I20260811 09:07:48.532431 768 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.131:45057
I20260811 09:07:48.532454 1047 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.131:45057 every 8 connection(s)
I20260811 09:07:48.534125 768 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439250192320-29290-0/minicluster-data/ts-2/data/info.pb
I20260811 09:07:48.536765 1048 heartbeater.cc:347] Connected to a master server at 127.28.154.190:35229
I20260811 09:07:48.536990 1048 heartbeater.cc:464] Registering TS with master...
I20260811 09:07:48.537655 1048 heartbeater.cc:511] Master 127.28.154.190:35229 requested a full tablet report, sending...
I20260811 09:07:48.539220 32567 ts_manager.cc:194] Registered new tserver with Master: 7136de9ce16f4703a928d1429952cdea (127.28.154.131:45057)
I20260811 09:07:48.540175 32567 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.154.131@KRBTEST.COM'} at 127.28.154.131:55569
I20260811 09:07:48.540465 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 768
I20260811 09:07:48.540635 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439250192320-29290-0/minicluster-data/ts-2/wal/instance
I20260811 09:07:48.541196 29290 external_mini_cluster.cc:1428] Setting key e9d30f0e10d193eaa7ec74ac7f93e2cf
I20260811 09:07:48.542531 32755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:48.526948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.131:45351 (local address 127.28.154.190:35229)
0811 09:07:48.527153 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:48.527160 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:48.528147 (+ 987us) server_negotiation.cc:407] Connection header received
0811 09:07:48.528394 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:48.528400 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:48.528480 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:48.528570 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:48.530294 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:48.531421 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:48.533505 (+ 2084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:48.534077 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:48.534183 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:48.536399 (+ 2216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:07:48.536406 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:07:48.536411 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:07:48.536447 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:07:48.539844 (+ 3397us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:48.540619 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:48.540625 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:48.540629 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:48.540766 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:48.541114 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:48.541118 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:48.541120 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:48.541368 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:07:48.541527 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:48.541950 (+ 423us) server_negotiation.cc:299] Negotiation successful
0811 09:07:48.542163 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 09:07:48.543391 1023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:48.527303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:35229 (local address 127.28.154.131:45351)
0811 09:07:48.528031 (+ 728us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:48.528056 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 09:07:48.528230 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:07:48.528619 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:07:48.528624 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:07:48.529398 (+ 774us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:07:48.530059 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:48.530079 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:48.531636 (+ 1557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:48.531642 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:48.533333 (+ 1691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:48.533351 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:48.534292 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:48.534300 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:48.534484 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:48.535256 (+ 772us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:07:48.535282 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:07:48.536205 (+ 923us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:07:48.539972 (+ 3767us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:07:48.539978 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:07:48.539982 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:07:48.540506 (+ 524us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:07:48.540870 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:07:48.540875 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:07:48.540879 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:07:48.541011 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:07:48.541491 (+ 480us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:07:48.541497 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:07:48.541665 (+ 168us) client_negotiation.cc:769] Sending connection context
0811 09:07:48.542696 (+ 1031us) client_negotiation.cc:240] Negotiation successful
0811 09:07:48.542966 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":559,"thread_start_us":419,"threads_started":1}
I20260811 09:07:48.544386 29290 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 09:07:48 dist-test-slave-xt4k krb5kdc[32478](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439268, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 09:07:48 dist-test-slave-xt4k krb5kdc[32478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439268, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.154.190@KRBTEST.COM
I20260811 09:07:48.596719 32755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:48.582336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47742 (local address 127.28.154.190:35229)
0811 09:07:48.582604 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:48.582611 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:48.582828 (+ 217us) server_negotiation.cc:407] Connection header received
0811 09:07:48.583024 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:48.583029 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:48.583110 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:48.583207 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:48.584702 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:48.585458 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:48.586694 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:48.587127 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:48.587240 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:48.591074 (+ 3834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:07:48.591079 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:07:48.591083 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:07:48.591121 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:07:48.594001 (+ 2880us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:48.594684 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:48.594689 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:48.594692 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:48.594828 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:48.595207 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:48.595211 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:48.595213 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:48.595523 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:07:48.595761 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:48.596077 (+ 316us) server_negotiation.cc:299] Negotiation successful
0811 09:07:48.596307 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260811 09:07:48.601017 32566 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47742:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 09:07:48.608124 32566 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 09:07:48.630924 1073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:48.621741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37208 (local address 127.28.154.131:45057)
0811 09:07:48.622481 (+ 740us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:48.622490 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:48.622526 (+ 36us) server_negotiation.cc:407] Connection header received
0811 09:07:48.622650 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:48.622658 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:48.622921 (+ 263us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:48.623130 (+ 209us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:07:48.624250 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:48.625899 (+ 1649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:48.628328 (+ 2429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:48.629797 (+ 1469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:48.629983 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:48.630187 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:48.630426 (+ 239us) server_negotiation.cc:299] Negotiation successful
0811 09:07:48.630585 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":543,"thread_start_us":629,"threads_started":1}
I20260811 09:07:48.630929 1067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:48.621284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.131:45057 (local address 127.0.0.1:37208)
0811 09:07:48.622200 (+ 916us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:48.622249 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 09:07:48.622437 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:07:48.623483 (+ 1046us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:07:48.623491 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:07:48.623522 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:07:48.623961 (+ 439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:48.623980 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:48.626028 (+ 2048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:48.626033 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:48.628142 (+ 2109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:48.628160 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:48.629983 (+ 1823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:48.629989 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:48.630190 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:48.630244 (+ 54us) client_negotiation.cc:769] Sending connection context
0811 09:07:48.630369 (+ 125us) client_negotiation.cc:240] Negotiation successful
0811 09:07:48.630571 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":648,"thread_start_us":510,"threads_started":1}
I20260811 09:07:48.631481 1075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:48.622606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.130:38009 (local address 127.0.0.1:59830)
0811 09:07:48.623178 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:48.623203 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 09:07:48.623335 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:07:48.624403 (+ 1068us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:07:48.624408 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:07:48.624432 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:07:48.624816 (+ 384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:48.624829 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:48.626722 (+ 1893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:48.626727 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:48.628939 (+ 2212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:48.628952 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:48.630745 (+ 1793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:48.630751 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:48.630906 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:48.630945 (+ 39us) client_negotiation.cc:769] Sending connection context
0811 09:07:48.631013 (+ 68us) client_negotiation.cc:240] Negotiation successful
0811 09:07:48.631149 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":275,"threads_started":1}
I20260811 09:07:48.632202 1077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:48.623223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.129:41515 (local address 127.0.0.1:49252)
0811 09:07:48.624177 (+ 954us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:48.624199 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 09:07:48.624302 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:07:48.625354 (+ 1052us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:07:48.625358 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:07:48.625383 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:07:48.625736 (+ 353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:48.625747 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:48.627491 (+ 1744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:48.627496 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:48.629507 (+ 2011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:48.629524 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:48.631418 (+ 1894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:48.631424 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:48.631615 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:48.631648 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 09:07:48.631715 (+ 67us) client_negotiation.cc:240] Negotiation successful
0811 09:07:48.631831 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":849,"mutex_wait_us":1,"thread_start_us":686,"threads_started":1}
I20260811 09:07:48.632767 1076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:48.622930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59830 (local address 127.28.154.130:38009)
0811 09:07:48.623804 (+ 874us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:48.623812 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:48.623840 (+ 28us) server_negotiation.cc:407] Connection header received
0811 09:07:48.623933 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:48.623940 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:48.624167 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:48.625000 (+ 833us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:07:48.625200 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:48.626587 (+ 1387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:48.629134 (+ 2547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:48.630451 (+ 1317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:48.631880 (+ 1429us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:48.632060 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:48.632143 (+ 83us) server_negotiation.cc:299] Negotiation successful
0811 09:07:48.632465 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":731,"thread_start_us":598,"threads_started":1}
I20260811 09:07:48.633172 1080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:48.623732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49252 (local address 127.28.154.129:41515)
0811 09:07:48.624799 (+ 1067us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:48.624807 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:48.624833 (+ 26us) server_negotiation.cc:407] Connection header received
0811 09:07:48.624938 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:48.624944 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:48.625144 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:48.625948 (+ 804us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:07:48.626138 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:48.627348 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:48.629681 (+ 2333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:48.631191 (+ 1510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:48.632525 (+ 1334us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:48.632691 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:48.632763 (+ 72us) server_negotiation.cc:299] Negotiation successful
0811 09:07:48.632930 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":907,"thread_start_us":777,"threads_started":1}
I20260811 09:07:48.635612 910 tablet_service.cc:1467] Processing CreateTablet for tablet 6a96ff9f1a9e4005ace3f929022acf40 (DEFAULT_TABLE table=test-table [id=2a3cf6dca65843dc9d483a8007c19471]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:07:48.637269 910 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a96ff9f1a9e4005ace3f929022acf40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:07:48.636971 644 tablet_service.cc:1467] Processing CreateTablet for tablet 6a96ff9f1a9e4005ace3f929022acf40 (DEFAULT_TABLE table=test-table [id=2a3cf6dca65843dc9d483a8007c19471]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:07:48.638303 644 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a96ff9f1a9e4005ace3f929022acf40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:07:48.642369 361 tablet_service.cc:1467] Processing CreateTablet for tablet 6a96ff9f1a9e4005ace3f929022acf40 (DEFAULT_TABLE table=test-table [id=2a3cf6dca65843dc9d483a8007c19471]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:07:48.643481 361 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a96ff9f1a9e4005ace3f929022acf40. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:07:48.654932 1085 tablet_bootstrap.cc:492] T 6a96ff9f1a9e4005ace3f929022acf40 P b66124a6ca6244748b38ebb07c6ca40e: Bootstrap starting.
I20260811 09:07:48.658649 1085 tablet_bootstrap.cc:654] T 6a96ff9f1a9e4005ace3f929022acf40 P b66124a6ca6244748b38ebb07c6ca40e: Neither blocks nor log segments found. Creating new log.
I20260811 09:07:48.658639 1083 tablet_bootstrap.cc:492] T 6a96ff9f1a9e4005ace3f929022acf40 P 7136de9ce16f4703a928d1429952cdea: Bootstrap starting.
I20260811 09:07:48.660547 1082 tablet_bootstrap.cc:492] T 6a96ff9f1a9e4005ace3f929022acf40 P e8b65b15973c4d7d896a96567f97caee: Bootstrap starting.
I20260811 09:07:48.661851 1085 log.cc:824] T 6a96ff9f1a9e4005ace3f929022acf40 P b66124a6ca6244748b38ebb07c6ca40e: Log is configured to *not* fsync() on all Append() calls
I20260811 09:07:48.662436 1083 tablet_bootstrap.cc:654] T 6a96ff9f1a9e4005ace3f929022acf40 P 7136de9ce16f4703a928d1429952cdea: Neither blocks nor log segments found. Creating new log.
I20260811 09:07:48.663810 1083 log.cc:824] T 6a96ff9f1a9e4005ace3f929022acf40 P 7136de9ce16f4703a928d1429952cdea: Log is configured to *not* fsync() on all Append() calls
I20260811 09:07:48.664032 1082 tablet_bootstrap.cc:654] T 6a96ff9f1a9e4005ace3f929022acf40 P e8b65b15973c4d7d896a96567f97caee: Neither blocks nor log segments found. Creating new log.
I20260811 09:07:48.665328 1082 log.cc:824] T 6a96ff9f1a9e4005ace3f929022acf40 P e8b65b15973c4d7d896a96567f97caee: Log is configured to *not* fsync() on all Append() calls
I20260811 09:07:48.665490 1085 tablet_bootstrap.cc:492] T 6a96ff9f1a9e4005ace3f929022acf40 P b66124a6ca6244748b38ebb07c6ca40e: No bootstrap required, opened a new log
I20260811 09:07:48.665807 1085 ts_tablet_manager.cc:1397] T 6a96ff9f1a9e4005ace3f929022acf40 P b66124a6ca6244748b38ebb07c6ca40e: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.000s
I20260811 09:07:48.666613 1083 tablet_bootstrap.cc:492] T 6a96ff9f1a9e4005ace3f929022acf40 P 7136de9ce16f4703a928d1429952cdea: No bootstrap required, opened a new log
I20260811 09:07:48.666903 1083 ts_tablet_manager.cc:1397] T 6a96ff9f1a9e4005ace3f929022acf40 P 7136de9ce16f4703a928d1429952cdea: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260811 09:07:48.668329 1082 tablet_bootstrap.cc:492] T 6a96ff9f1a9e4005ace3f929022acf40 P e8b65b15973c4d7d896a96567f97caee: No bootstrap required, opened a new log
I20260811 09:07:48.668620 1082 ts_tablet_manager.cc:1397] T 6a96ff9f1a9e4005ace3f929022acf40 P e8b65b15973c4d7d896a96567f97caee: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260811 09:07:48.674814 1083 raft_consensus.cc:348] T 6a96ff9f1a9e4005ace3f929022acf40 P 7136de9ce16f4703a928d1429952cdea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7136de9ce16f4703a928d1429952cdea" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 45057 } } peers { permanent_uuid: "e8b65b15973c4d7d896a96567f97caee" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 38009 } } peers { permanent_uuid: "b66124a6ca6244748b38ebb07c6ca40e" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41515 } }
I20260811 09:07:48.675220 1083 raft_consensus.cc:374] T 6a96ff9f1a9e4005ace3f929022acf40 P 7136de9ce16f4703a928d1429952cdea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:07:48.675341 1083 raft_consensus.cc:729] T 6a96ff9f1a9e4005ace3f929022acf40 P 7136de9ce16f4703a928d1429952cdea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7136de9ce16f4703a928d1429952cdea, State: Initialized, Role: FOLLOWER
I20260811 09:07:48.675942 1083 consensus_queue.cc:260] T 6a96ff9f1a9e4005ace3f929022acf40 P 7136de9ce16f4703a928d1429952cdea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7136de9ce16f4703a928d1429952cdea" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 45057 } } peers { permanent_uuid: "e8b65b15973c4d7d896a96567f97caee" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 38009 } } peers { permanent_uuid: "b66124a6ca6244748b38ebb07c6ca40e" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41515 } }
I20260811 09:07:48.677071 1048 heartbeater.cc:503] Master 127.28.154.190:35229 was elected leader, sending a full tablet report...
I20260811 09:07:48.676841 1085 raft_consensus.cc:348] T 6a96ff9f1a9e4005ace3f929022acf40 P b66124a6ca6244748b38ebb07c6ca40e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7136de9ce16f4703a928d1429952cdea" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 45057 } } peers { permanent_uuid: "e8b65b15973c4d7d896a96567f97caee" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 38009 } } peers { permanent_uuid: "b66124a6ca6244748b38ebb07c6ca40e" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41515 } }
I20260811 09:07:48.677315 1085 raft_consensus.cc:374] T 6a96ff9f1a9e4005ace3f929022acf40 P b66124a6ca6244748b38ebb07c6ca40e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:07:48.677423 1085 raft_consensus.cc:729] T 6a96ff9f1a9e4005ace3f929022acf40 P b66124a6ca6244748b38ebb07c6ca40e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b66124a6ca6244748b38ebb07c6ca40e, State: Initialized, Role: FOLLOWER
I20260811 09:07:48.678072 1085 consensus_queue.cc:260] T 6a96ff9f1a9e4005ace3f929022acf40 P b66124a6ca6244748b38ebb07c6ca40e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7136de9ce16f4703a928d1429952cdea" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 45057 } } peers { permanent_uuid: "e8b65b15973c4d7d896a96567f97caee" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 38009 } } peers { permanent_uuid: "b66124a6ca6244748b38ebb07c6ca40e" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41515 } }
I20260811 09:07:48.679319 1082 raft_consensus.cc:348] T 6a96ff9f1a9e4005ace3f929022acf40 P e8b65b15973c4d7d896a96567f97caee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7136de9ce16f4703a928d1429952cdea" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 45057 } } peers { permanent_uuid: "e8b65b15973c4d7d896a96567f97caee" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 38009 } } peers { permanent_uuid: "b66124a6ca6244748b38ebb07c6ca40e" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41515 } }
I20260811 09:07:48.679883 1082 raft_consensus.cc:374] T 6a96ff9f1a9e4005ace3f929022acf40 P e8b65b15973c4d7d896a96567f97caee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:07:48.680152 1085 ts_tablet_manager.cc:1428] T 6a96ff9f1a9e4005ace3f929022acf40 P b66124a6ca6244748b38ebb07c6ca40e: Time spent starting tablet: real 0.014s user 0.012s sys 0.000s
I20260811 09:07:48.680143 1082 raft_consensus.cc:729] T 6a96ff9f1a9e4005ace3f929022acf40 P e8b65b15973c4d7d896a96567f97caee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8b65b15973c4d7d896a96567f97caee, State: Initialized, Role: FOLLOWER
I20260811 09:07:48.680821 1082 consensus_queue.cc:260] T 6a96ff9f1a9e4005ace3f929022acf40 P e8b65b15973c4d7d896a96567f97caee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7136de9ce16f4703a928d1429952cdea" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 45057 } } peers { permanent_uuid: "e8b65b15973c4d7d896a96567f97caee" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 38009 } } peers { permanent_uuid: "b66124a6ca6244748b38ebb07c6ca40e" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41515 } }
I20260811 09:07:48.681896 761 heartbeater.cc:503] Master 127.28.154.190:35229 was elected leader, sending a full tablet report...
I20260811 09:07:48.682559 1082 ts_tablet_manager.cc:1428] T 6a96ff9f1a9e4005ace3f929022acf40 P e8b65b15973c4d7d896a96567f97caee: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260811 09:07:48.682554 1083 ts_tablet_manager.cc:1428] T 6a96ff9f1a9e4005ace3f929022acf40 P 7136de9ce16f4703a928d1429952cdea: Time spent starting tablet: real 0.016s user 0.009s sys 0.002s
I20260811 09:07:48.690234 1096 raft_consensus.cc:482] T 6a96ff9f1a9e4005ace3f929022acf40 P 7136de9ce16f4703a928d1429952cdea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:07:48.690546 1096 raft_consensus.cc:504] T 6a96ff9f1a9e4005ace3f929022acf40 P 7136de9ce16f4703a928d1429952cdea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7136de9ce16f4703a928d1429952cdea" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 45057 } } peers { permanent_uuid: "e8b65b15973c4d7d896a96567f97caee" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 38009 } } peers { permanent_uuid: "b66124a6ca6244748b38ebb07c6ca40e" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41515 } }
I20260811 09:07:48.692201 1096 leader_election.cc:288] T 6a96ff9f1a9e4005ace3f929022acf40 P 7136de9ce16f4703a928d1429952cdea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8b65b15973c4d7d896a96567f97caee (127.28.154.130:38009), b66124a6ca6244748b38ebb07c6ca40e (127.28.154.129:41515)
I20260811 09:07:48.697949 1076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:48.692717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.131:44083 (local address 127.28.154.130:38009)
0811 09:07:48.692978 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:48.692984 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:48.693007 (+ 23us) server_negotiation.cc:407] Connection header received
0811 09:07:48.693114 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:48.693119 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:48.693199 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:48.693308 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:07:48.693815 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:48.694756 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:48.696274 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:48.697105 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:48.697253 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:48.697328 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:48.697579 (+ 251us) server_negotiation.cc:299] Negotiation successful
0811 09:07:48.697710 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260811 09:07:48.697942 820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:48.692672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.130:38009 (local address 127.28.154.131:44083)
0811 09:07:48.692877 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:48.692894 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 09:07:48.692996 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:07:48.693445 (+ 449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:07:48.693450 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:07:48.693480 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:07:48.693663 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:48.693672 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:48.694876 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:48.694882 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:48.696113 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:48.696126 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:48.697246 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:48.697252 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:48.697423 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:48.697448 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 09:07:48.697531 (+ 83us) client_negotiation.cc:240] Negotiation successful
0811 09:07:48.697665 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63}
I20260811 09:07:48.699218 1100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:48.693108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.129:41515 (local address 127.28.154.131:42019)
0811 09:07:48.693977 (+ 869us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:48.693990 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 09:07:48.694090 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:07:48.694426 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:07:48.694429 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:07:48.694453 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:07:48.694754 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:48.694763 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:48.695769 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:48.695773 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:48.697134 (+ 1361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:48.697147 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:48.698443 (+ 1296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:48.698448 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:48.698598 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:48.698634 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 09:07:48.698816 (+ 182us) client_negotiation.cc:240] Negotiation successful
0811 09:07:48.698947 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":752,"thread_start_us":570,"threads_started":1}
I20260811 09:07:48.699203 1080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:48.693365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.131:42019 (local address 127.28.154.129:41515)
0811 09:07:48.693958 (+ 593us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:48.693963 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:48.694055 (+ 92us) server_negotiation.cc:407] Connection header received
0811 09:07:48.694195 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:48.694200 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:48.694284 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:48.694389 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:07:48.694928 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:48.695635 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:48.697290 (+ 1655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:48.698214 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:48.698382 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:48.698469 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:48.698850 (+ 381us) server_negotiation.cc:299] Negotiation successful
0811 09:07:48.698964 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":432}
I20260811 09:07:48.699620 687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a96ff9f1a9e4005ace3f929022acf40" candidate_uuid: "7136de9ce16f4703a928d1429952cdea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8b65b15973c4d7d896a96567f97caee" is_pre_election: true
I20260811 09:07:48.700418 687 raft_consensus.cc:2454] T 6a96ff9f1a9e4005ace3f929022acf40 P e8b65b15973c4d7d896a96567f97caee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7136de9ce16f4703a928d1429952cdea in term 0.
I20260811 09:07:48.700460 409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a96ff9f1a9e4005ace3f929022acf40" candidate_uuid: "7136de9ce16f4703a928d1429952cdea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b66124a6ca6244748b38ebb07c6ca40e" is_pre_election: true
I20260811 09:07:48.701086 409 raft_consensus.cc:2454] T 6a96ff9f1a9e4005ace3f929022acf40 P b66124a6ca6244748b38ebb07c6ca40e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7136de9ce16f4703a928d1429952cdea in term 0.
I20260811 09:07:48.701318 802 leader_election.cc:302] T 6a96ff9f1a9e4005ace3f929022acf40 P 7136de9ce16f4703a928d1429952cdea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7136de9ce16f4703a928d1429952cdea, e8b65b15973c4d7d896a96567f97caee; no voters:
I20260811 09:07:48.701756 1096 raft_consensus.cc:2781] T 6a96ff9f1a9e4005ace3f929022acf40 P 7136de9ce16f4703a928d1429952cdea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 09:07:48.701934 1096 raft_consensus.cc:482] T 6a96ff9f1a9e4005ace3f929022acf40 P 7136de9ce16f4703a928d1429952cdea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 09:07:48.702044 1096 raft_consensus.cc:3037] T 6a96ff9f1a9e4005ace3f929022acf40 P 7136de9ce16f4703a928d1429952cdea [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:07:48.704432 1096 raft_consensus.cc:504] T 6a96ff9f1a9e4005ace3f929022acf40 P 7136de9ce16f4703a928d1429952cdea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7136de9ce16f4703a928d1429952cdea" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 45057 } } peers { permanent_uuid: "e8b65b15973c4d7d896a96567f97caee" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 38009 } } peers { permanent_uuid: "b66124a6ca6244748b38ebb07c6ca40e" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41515 } }
I20260811 09:07:48.705024 1096 leader_election.cc:288] T 6a96ff9f1a9e4005ace3f929022acf40 P 7136de9ce16f4703a928d1429952cdea [CANDIDATE]: Term 1 election: Requested vote from peers e8b65b15973c4d7d896a96567f97caee (127.28.154.130:38009), b66124a6ca6244748b38ebb07c6ca40e (127.28.154.129:41515)
I20260811 09:07:48.705546 687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a96ff9f1a9e4005ace3f929022acf40" candidate_uuid: "7136de9ce16f4703a928d1429952cdea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8b65b15973c4d7d896a96567f97caee"
I20260811 09:07:48.705632 409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a96ff9f1a9e4005ace3f929022acf40" candidate_uuid: "7136de9ce16f4703a928d1429952cdea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b66124a6ca6244748b38ebb07c6ca40e"
I20260811 09:07:48.705782 687 raft_consensus.cc:3037] T 6a96ff9f1a9e4005ace3f929022acf40 P e8b65b15973c4d7d896a96567f97caee [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:07:48.705838 409 raft_consensus.cc:3037] T 6a96ff9f1a9e4005ace3f929022acf40 P b66124a6ca6244748b38ebb07c6ca40e [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:07:48.709070 409 raft_consensus.cc:2454] T 6a96ff9f1a9e4005ace3f929022acf40 P b66124a6ca6244748b38ebb07c6ca40e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7136de9ce16f4703a928d1429952cdea in term 1.
I20260811 09:07:48.709072 687 raft_consensus.cc:2454] T 6a96ff9f1a9e4005ace3f929022acf40 P e8b65b15973c4d7d896a96567f97caee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7136de9ce16f4703a928d1429952cdea in term 1.
I20260811 09:07:48.709477 802 leader_election.cc:302] T 6a96ff9f1a9e4005ace3f929022acf40 P 7136de9ce16f4703a928d1429952cdea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7136de9ce16f4703a928d1429952cdea, e8b65b15973c4d7d896a96567f97caee; no voters:
I20260811 09:07:48.709748 1096 raft_consensus.cc:2781] T 6a96ff9f1a9e4005ace3f929022acf40 P 7136de9ce16f4703a928d1429952cdea [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:07:48.710170 1096 raft_consensus.cc:686] T 6a96ff9f1a9e4005ace3f929022acf40 P 7136de9ce16f4703a928d1429952cdea [term 1 LEADER]: Becoming Leader. State: Replica: 7136de9ce16f4703a928d1429952cdea, State: Running, Role: LEADER
I20260811 09:07:48.710659 1096 consensus_queue.cc:237] T 6a96ff9f1a9e4005ace3f929022acf40 P 7136de9ce16f4703a928d1429952cdea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7136de9ce16f4703a928d1429952cdea" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 45057 } } peers { permanent_uuid: "e8b65b15973c4d7d896a96567f97caee" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 38009 } } peers { permanent_uuid: "b66124a6ca6244748b38ebb07c6ca40e" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41515 } }
W20260811 09:07:48.713068 510 tablet.cc:2369] T 6a96ff9f1a9e4005ace3f929022acf40 P b66124a6ca6244748b38ebb07c6ca40e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 09:07:48.715553 32563 catalog_manager.cc:5495] T 6a96ff9f1a9e4005ace3f929022acf40 P 7136de9ce16f4703a928d1429952cdea reported cstate change: term changed from 0 to 1, leader changed from <none> to 7136de9ce16f4703a928d1429952cdea (127.28.154.131). New cstate: current_term: 1 leader_uuid: "7136de9ce16f4703a928d1429952cdea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7136de9ce16f4703a928d1429952cdea" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 45057 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e8b65b15973c4d7d896a96567f97caee" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 38009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b66124a6ca6244748b38ebb07c6ca40e" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41515 } health_report { overall_health: UNKNOWN } } }
W20260811 09:07:48.747112 763 tablet.cc:2369] T 6a96ff9f1a9e4005ace3f929022acf40 P e8b65b15973c4d7d896a96567f97caee: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 11 09:07:48 dist-test-slave-xt4k krb5kdc[32478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439266, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.190@KRBTEST.COM for kudu/127.28.154.190@KRBTEST.COM
I20260811 09:07:48.780624 1124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:48.767057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:35229 (local address 127.0.0.1:47754)
0811 09:07:48.767696 (+ 639us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:48.767713 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 09:07:48.767849 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:07:48.768194 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:07:48.768197 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:07:48.769006 (+ 809us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:07:48.769302 (+ 296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:48.769312 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:48.770369 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:48.770374 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:48.771385 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:48.771398 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:48.772185 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:48.772190 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:48.772307 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:48.773046 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:07:48.773064 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:07:48.775295 (+ 2231us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:07:48.778124 (+ 2829us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:07:48.778136 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:07:48.778142 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:07:48.778565 (+ 423us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:07:48.778910 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:07:48.778914 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:07:48.778916 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:07:48.779065 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:07:48.779602 (+ 537us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:07:48.779611 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:07:48.779803 (+ 192us) client_negotiation.cc:769] Sending connection context
0811 09:07:48.779960 (+ 157us) client_negotiation.cc:240] Negotiation successful
0811 09:07:48.780182 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":340,"threads_started":1}
I20260811 09:07:48.780687 32755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:48.767048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47754 (local address 127.28.154.190:35229)
0811 09:07:48.767250 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:48.767255 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:48.767799 (+ 544us) server_negotiation.cc:407] Connection header received
0811 09:07:48.768001 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:48.768006 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:48.768071 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:48.768161 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:48.769466 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:48.770241 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:48.771528 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:48.772003 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:48.772107 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:48.775501 (+ 3394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:07:48.775507 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:07:48.775510 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:07:48.775543 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:07:48.777986 (+ 2443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:48.778654 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:48.778657 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:48.778660 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:48.778810 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:48.779167 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:48.779171 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:48.779173 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:48.779471 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:07:48.779616 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:48.780112 (+ 496us) server_negotiation.cc:299] Negotiation successful
0811 09:07:48.780356 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
W20260811 09:07:48.786156 1051 tablet.cc:2369] T 6a96ff9f1a9e4005ace3f929022acf40 P 7136de9ce16f4703a928d1429952cdea: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 09:07:48.786144 32563 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.28.154.190@KRBTEST.COM'} at 127.0.0.1:47754:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 09:07:48.787297 32563 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 09:07:48.794579 361 tablet_service.cc:1467] Processing CreateTablet for tablet 9811bfa9f5ca489e953f18fb61cfaad9 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=754ab257512d40539e10ae987ce81bac]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 09:07:48.794641 644 tablet_service.cc:1467] Processing CreateTablet for tablet 9811bfa9f5ca489e953f18fb61cfaad9 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=754ab257512d40539e10ae987ce81bac]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 09:07:48.795764 361 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9811bfa9f5ca489e953f18fb61cfaad9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:07:48.795764 644 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9811bfa9f5ca489e953f18fb61cfaad9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:07:48.803264 1085 tablet_bootstrap.cc:492] T 9811bfa9f5ca489e953f18fb61cfaad9 P b66124a6ca6244748b38ebb07c6ca40e: Bootstrap starting.
I20260811 09:07:48.804584 1082 tablet_bootstrap.cc:492] T 9811bfa9f5ca489e953f18fb61cfaad9 P e8b65b15973c4d7d896a96567f97caee: Bootstrap starting.
I20260811 09:07:48.804536 910 tablet_service.cc:1467] Processing CreateTablet for tablet 9811bfa9f5ca489e953f18fb61cfaad9 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=754ab257512d40539e10ae987ce81bac]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 09:07:48.805428 910 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9811bfa9f5ca489e953f18fb61cfaad9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:07:48.805933 1082 tablet_bootstrap.cc:654] T 9811bfa9f5ca489e953f18fb61cfaad9 P e8b65b15973c4d7d896a96567f97caee: Neither blocks nor log segments found. Creating new log.
I20260811 09:07:48.807318 1085 tablet_bootstrap.cc:654] T 9811bfa9f5ca489e953f18fb61cfaad9 P b66124a6ca6244748b38ebb07c6ca40e: Neither blocks nor log segments found. Creating new log.
I20260811 09:07:48.809212 1082 tablet_bootstrap.cc:492] T 9811bfa9f5ca489e953f18fb61cfaad9 P e8b65b15973c4d7d896a96567f97caee: No bootstrap required, opened a new log
I20260811 09:07:48.809370 1082 ts_tablet_manager.cc:1397] T 9811bfa9f5ca489e953f18fb61cfaad9 P e8b65b15973c4d7d896a96567f97caee: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260811 09:07:48.809526 1085 tablet_bootstrap.cc:492] T 9811bfa9f5ca489e953f18fb61cfaad9 P b66124a6ca6244748b38ebb07c6ca40e: No bootstrap required, opened a new log
I20260811 09:07:48.809675 1085 ts_tablet_manager.cc:1397] T 9811bfa9f5ca489e953f18fb61cfaad9 P b66124a6ca6244748b38ebb07c6ca40e: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.000s
I20260811 09:07:48.810205 1082 raft_consensus.cc:348] T 9811bfa9f5ca489e953f18fb61cfaad9 P e8b65b15973c4d7d896a96567f97caee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b66124a6ca6244748b38ebb07c6ca40e" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41515 } } peers { permanent_uuid: "e8b65b15973c4d7d896a96567f97caee" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 38009 } } peers { permanent_uuid: "7136de9ce16f4703a928d1429952cdea" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 45057 } }
I20260811 09:07:48.810416 1085 raft_consensus.cc:348] T 9811bfa9f5ca489e953f18fb61cfaad9 P b66124a6ca6244748b38ebb07c6ca40e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b66124a6ca6244748b38ebb07c6ca40e" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41515 } } peers { permanent_uuid: "e8b65b15973c4d7d896a96567f97caee" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 38009 } } peers { permanent_uuid: "7136de9ce16f4703a928d1429952cdea" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 45057 } }
I20260811 09:07:48.810653 1085 raft_consensus.cc:374] T 9811bfa9f5ca489e953f18fb61cfaad9 P b66124a6ca6244748b38ebb07c6ca40e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:07:48.810750 1085 raft_consensus.cc:729] T 9811bfa9f5ca489e953f18fb61cfaad9 P b66124a6ca6244748b38ebb07c6ca40e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b66124a6ca6244748b38ebb07c6ca40e, State: Initialized, Role: FOLLOWER
I20260811 09:07:48.811075 1082 raft_consensus.cc:374] T 9811bfa9f5ca489e953f18fb61cfaad9 P e8b65b15973c4d7d896a96567f97caee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:07:48.810976 1085 consensus_queue.cc:260] T 9811bfa9f5ca489e953f18fb61cfaad9 P b66124a6ca6244748b38ebb07c6ca40e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b66124a6ca6244748b38ebb07c6ca40e" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41515 } } peers { permanent_uuid: "e8b65b15973c4d7d896a96567f97caee" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 38009 } } peers { permanent_uuid: "7136de9ce16f4703a928d1429952cdea" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 45057 } }
I20260811 09:07:48.811184 1082 raft_consensus.cc:729] T 9811bfa9f5ca489e953f18fb61cfaad9 P e8b65b15973c4d7d896a96567f97caee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8b65b15973c4d7d896a96567f97caee, State: Initialized, Role: FOLLOWER
I20260811 09:07:48.811472 1082 consensus_queue.cc:260] T 9811bfa9f5ca489e953f18fb61cfaad9 P e8b65b15973c4d7d896a96567f97caee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b66124a6ca6244748b38ebb07c6ca40e" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41515 } } peers { permanent_uuid: "e8b65b15973c4d7d896a96567f97caee" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 38009 } } peers { permanent_uuid: "7136de9ce16f4703a928d1429952cdea" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 45057 } }
I20260811 09:07:48.811636 1085 ts_tablet_manager.cc:1428] T 9811bfa9f5ca489e953f18fb61cfaad9 P b66124a6ca6244748b38ebb07c6ca40e: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260811 09:07:48.811522 1097 tablet_replica.cc:439] T 9811bfa9f5ca489e953f18fb61cfaad9 P b66124a6ca6244748b38ebb07c6ca40e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b66124a6ca6244748b38ebb07c6ca40e" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41515 } } peers { permanent_uuid: "e8b65b15973c4d7d896a96567f97caee" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 38009 } } peers { permanent_uuid: "7136de9ce16f4703a928d1429952cdea" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 45057 } } }
I20260811 09:07:48.811956 1097 tablet_replica.cc:442] T 9811bfa9f5ca489e953f18fb61cfaad9 P b66124a6ca6244748b38ebb07c6ca40e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 09:07:48.812129 1082 ts_tablet_manager.cc:1428] T 9811bfa9f5ca489e953f18fb61cfaad9 P e8b65b15973c4d7d896a96567f97caee: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260811 09:07:48.812180 1098 tablet_replica.cc:439] T 9811bfa9f5ca489e953f18fb61cfaad9 P e8b65b15973c4d7d896a96567f97caee: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b66124a6ca6244748b38ebb07c6ca40e" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41515 } } peers { permanent_uuid: "e8b65b15973c4d7d896a96567f97caee" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 38009 } } peers { permanent_uuid: "7136de9ce16f4703a928d1429952cdea" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 45057 } } }
I20260811 09:07:48.812564 1098 tablet_replica.cc:442] T 9811bfa9f5ca489e953f18fb61cfaad9 P e8b65b15973c4d7d896a96567f97caee: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 09:07:48.813160 1083 tablet_bootstrap.cc:492] T 9811bfa9f5ca489e953f18fb61cfaad9 P 7136de9ce16f4703a928d1429952cdea: Bootstrap starting.
I20260811 09:07:48.814790 1083 tablet_bootstrap.cc:654] T 9811bfa9f5ca489e953f18fb61cfaad9 P 7136de9ce16f4703a928d1429952cdea: Neither blocks nor log segments found. Creating new log.
I20260811 09:07:48.818035 1083 tablet_bootstrap.cc:492] T 9811bfa9f5ca489e953f18fb61cfaad9 P 7136de9ce16f4703a928d1429952cdea: No bootstrap required, opened a new log
I20260811 09:07:48.818188 1083 ts_tablet_manager.cc:1397] T 9811bfa9f5ca489e953f18fb61cfaad9 P 7136de9ce16f4703a928d1429952cdea: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.002s
I20260811 09:07:48.818969 1083 raft_consensus.cc:348] T 9811bfa9f5ca489e953f18fb61cfaad9 P 7136de9ce16f4703a928d1429952cdea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b66124a6ca6244748b38ebb07c6ca40e" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41515 } } peers { permanent_uuid: "e8b65b15973c4d7d896a96567f97caee" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 38009 } } peers { permanent_uuid: "7136de9ce16f4703a928d1429952cdea" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 45057 } }
I20260811 09:07:48.819171 1083 raft_consensus.cc:374] T 9811bfa9f5ca489e953f18fb61cfaad9 P 7136de9ce16f4703a928d1429952cdea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:07:48.819260 1083 raft_consensus.cc:729] T 9811bfa9f5ca489e953f18fb61cfaad9 P 7136de9ce16f4703a928d1429952cdea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7136de9ce16f4703a928d1429952cdea, State: Initialized, Role: FOLLOWER
I20260811 09:07:48.819465 1083 consensus_queue.cc:260] T 9811bfa9f5ca489e953f18fb61cfaad9 P 7136de9ce16f4703a928d1429952cdea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b66124a6ca6244748b38ebb07c6ca40e" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41515 } } peers { permanent_uuid: "e8b65b15973c4d7d896a96567f97caee" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 38009 } } peers { permanent_uuid: "7136de9ce16f4703a928d1429952cdea" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 45057 } }
I20260811 09:07:48.819886 1096 tablet_replica.cc:439] T 9811bfa9f5ca489e953f18fb61cfaad9 P 7136de9ce16f4703a928d1429952cdea: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b66124a6ca6244748b38ebb07c6ca40e" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41515 } } peers { permanent_uuid: "e8b65b15973c4d7d896a96567f97caee" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 38009 } } peers { permanent_uuid: "7136de9ce16f4703a928d1429952cdea" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 45057 } } }
I20260811 09:07:48.820048 1083 ts_tablet_manager.cc:1428] T 9811bfa9f5ca489e953f18fb61cfaad9 P 7136de9ce16f4703a928d1429952cdea: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260811 09:07:48.820180 1096 tablet_replica.cc:442] T 9811bfa9f5ca489e953f18fb61cfaad9 P 7136de9ce16f4703a928d1429952cdea: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 09:07:48.949234 1096 raft_consensus.cc:482] T 9811bfa9f5ca489e953f18fb61cfaad9 P 7136de9ce16f4703a928d1429952cdea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:07:48.949527 1096 raft_consensus.cc:504] T 9811bfa9f5ca489e953f18fb61cfaad9 P 7136de9ce16f4703a928d1429952cdea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b66124a6ca6244748b38ebb07c6ca40e" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41515 } } peers { permanent_uuid: "e8b65b15973c4d7d896a96567f97caee" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 38009 } } peers { permanent_uuid: "7136de9ce16f4703a928d1429952cdea" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 45057 } }
I20260811 09:07:48.950141 1096 leader_election.cc:288] T 9811bfa9f5ca489e953f18fb61cfaad9 P 7136de9ce16f4703a928d1429952cdea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b66124a6ca6244748b38ebb07c6ca40e (127.28.154.129:41515), e8b65b15973c4d7d896a96567f97caee (127.28.154.130:38009)
I20260811 09:07:48.950852 409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9811bfa9f5ca489e953f18fb61cfaad9" candidate_uuid: "7136de9ce16f4703a928d1429952cdea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b66124a6ca6244748b38ebb07c6ca40e" is_pre_election: true
I20260811 09:07:48.951031 687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9811bfa9f5ca489e953f18fb61cfaad9" candidate_uuid: "7136de9ce16f4703a928d1429952cdea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8b65b15973c4d7d896a96567f97caee" is_pre_election: true
I20260811 09:07:48.951182 409 raft_consensus.cc:2454] T 9811bfa9f5ca489e953f18fb61cfaad9 P b66124a6ca6244748b38ebb07c6ca40e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7136de9ce16f4703a928d1429952cdea in term 0.
I20260811 09:07:48.951304 687 raft_consensus.cc:2454] T 9811bfa9f5ca489e953f18fb61cfaad9 P e8b65b15973c4d7d896a96567f97caee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7136de9ce16f4703a928d1429952cdea in term 0.
I20260811 09:07:48.951676 802 leader_election.cc:302] T 9811bfa9f5ca489e953f18fb61cfaad9 P 7136de9ce16f4703a928d1429952cdea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7136de9ce16f4703a928d1429952cdea, b66124a6ca6244748b38ebb07c6ca40e; no voters:
I20260811 09:07:48.951970 1096 raft_consensus.cc:2781] T 9811bfa9f5ca489e953f18fb61cfaad9 P 7136de9ce16f4703a928d1429952cdea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 09:07:48.952075 1096 raft_consensus.cc:482] T 9811bfa9f5ca489e953f18fb61cfaad9 P 7136de9ce16f4703a928d1429952cdea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 09:07:48.952184 1096 raft_consensus.cc:3037] T 9811bfa9f5ca489e953f18fb61cfaad9 P 7136de9ce16f4703a928d1429952cdea [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:07:48.954272 1096 raft_consensus.cc:504] T 9811bfa9f5ca489e953f18fb61cfaad9 P 7136de9ce16f4703a928d1429952cdea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b66124a6ca6244748b38ebb07c6ca40e" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41515 } } peers { permanent_uuid: "e8b65b15973c4d7d896a96567f97caee" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 38009 } } peers { permanent_uuid: "7136de9ce16f4703a928d1429952cdea" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 45057 } }
I20260811 09:07:48.954873 1096 leader_election.cc:288] T 9811bfa9f5ca489e953f18fb61cfaad9 P 7136de9ce16f4703a928d1429952cdea [CANDIDATE]: Term 1 election: Requested vote from peers b66124a6ca6244748b38ebb07c6ca40e (127.28.154.129:41515), e8b65b15973c4d7d896a96567f97caee (127.28.154.130:38009)
I20260811 09:07:48.955449 687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9811bfa9f5ca489e953f18fb61cfaad9" candidate_uuid: "7136de9ce16f4703a928d1429952cdea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8b65b15973c4d7d896a96567f97caee"
I20260811 09:07:48.955505 409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9811bfa9f5ca489e953f18fb61cfaad9" candidate_uuid: "7136de9ce16f4703a928d1429952cdea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b66124a6ca6244748b38ebb07c6ca40e"
I20260811 09:07:48.955693 687 raft_consensus.cc:3037] T 9811bfa9f5ca489e953f18fb61cfaad9 P e8b65b15973c4d7d896a96567f97caee [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:07:48.955786 409 raft_consensus.cc:3037] T 9811bfa9f5ca489e953f18fb61cfaad9 P b66124a6ca6244748b38ebb07c6ca40e [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:07:48.957943 409 raft_consensus.cc:2454] T 9811bfa9f5ca489e953f18fb61cfaad9 P b66124a6ca6244748b38ebb07c6ca40e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7136de9ce16f4703a928d1429952cdea in term 1.
I20260811 09:07:48.958112 687 raft_consensus.cc:2454] T 9811bfa9f5ca489e953f18fb61cfaad9 P e8b65b15973c4d7d896a96567f97caee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7136de9ce16f4703a928d1429952cdea in term 1.
I20260811 09:07:48.958428 802 leader_election.cc:302] T 9811bfa9f5ca489e953f18fb61cfaad9 P 7136de9ce16f4703a928d1429952cdea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7136de9ce16f4703a928d1429952cdea, b66124a6ca6244748b38ebb07c6ca40e; no voters:
I20260811 09:07:48.958726 1096 raft_consensus.cc:2781] T 9811bfa9f5ca489e953f18fb61cfaad9 P 7136de9ce16f4703a928d1429952cdea [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:07:48.958899 1096 raft_consensus.cc:686] T 9811bfa9f5ca489e953f18fb61cfaad9 P 7136de9ce16f4703a928d1429952cdea [term 1 LEADER]: Becoming Leader. State: Replica: 7136de9ce16f4703a928d1429952cdea, State: Running, Role: LEADER
I20260811 09:07:48.959147 1096 consensus_queue.cc:237] T 9811bfa9f5ca489e953f18fb61cfaad9 P 7136de9ce16f4703a928d1429952cdea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b66124a6ca6244748b38ebb07c6ca40e" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41515 } } peers { permanent_uuid: "e8b65b15973c4d7d896a96567f97caee" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 38009 } } peers { permanent_uuid: "7136de9ce16f4703a928d1429952cdea" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 45057 } }
I20260811 09:07:48.960948 1101 tablet_replica.cc:439] T 9811bfa9f5ca489e953f18fb61cfaad9 P 7136de9ce16f4703a928d1429952cdea: TxnStatusTablet state changed. Reason: New leader 7136de9ce16f4703a928d1429952cdea. Latest consensus state: current_term: 1 leader_uuid: "7136de9ce16f4703a928d1429952cdea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b66124a6ca6244748b38ebb07c6ca40e" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41515 } } peers { permanent_uuid: "e8b65b15973c4d7d896a96567f97caee" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 38009 } } peers { permanent_uuid: "7136de9ce16f4703a928d1429952cdea" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 45057 } } }
I20260811 09:07:48.961321 1101 tablet_replica.cc:442] T 9811bfa9f5ca489e953f18fb61cfaad9 P 7136de9ce16f4703a928d1429952cdea: This TxnStatusTablet replica's current role is: LEADER
I20260811 09:07:48.962070 1129 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 09:07:48.962332 1129 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 09:07:48.962119 32563 catalog_manager.cc:5495] T 9811bfa9f5ca489e953f18fb61cfaad9 P 7136de9ce16f4703a928d1429952cdea reported cstate change: term changed from 0 to 1, leader changed from <none> to 7136de9ce16f4703a928d1429952cdea (127.28.154.131). New cstate: current_term: 1 leader_uuid: "7136de9ce16f4703a928d1429952cdea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b66124a6ca6244748b38ebb07c6ca40e" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8b65b15973c4d7d896a96567f97caee" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 38009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7136de9ce16f4703a928d1429952cdea" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 45057 } health_report { overall_health: HEALTHY } } }
I20260811 09:07:48.983512 1124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:48.976863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.131:45057 (local address 127.0.0.1:37218)
0811 09:07:48.977107 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:48.977126 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 09:07:48.977247 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:07:48.977725 (+ 478us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:07:48.977728 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:07:48.977738 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 09:07:48.977908 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:48.977915 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:48.979421 (+ 1506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:48.979456 (+ 35us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:48.980652 (+ 1196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:48.980668 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:48.981674 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:48.981678 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:48.981833 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:48.981897 (+ 64us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 09:07:48.983029 (+ 1132us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 09:07:48.983056 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 09:07:48.983136 (+ 80us) client_negotiation.cc:240] Negotiation successful
0811 09:07:48.983284 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260811 09:07:48.983626 1073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:48.976859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37218 (local address 127.28.154.131:45057)
0811 09:07:48.977090 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:48.977107 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:48.977190 (+ 83us) server_negotiation.cc:407] Connection header received
0811 09:07:48.977440 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:48.977446 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:48.977559 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:48.977746 (+ 187us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:07:48.978030 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:48.979241 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:48.980813 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:48.981517 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:48.981662 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:48.982112 (+ 450us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:07:48.982896 (+ 784us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:07:48.983062 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:48.983162 (+ 100us) server_negotiation.cc:299] Negotiation successful
0811 09:07:48.983338 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 09:07:48.992820 687 raft_consensus.cc:1261] T 9811bfa9f5ca489e953f18fb61cfaad9 P e8b65b15973c4d7d896a96567f97caee [term 1 FOLLOWER]: Refusing update from remote peer 7136de9ce16f4703a928d1429952cdea: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:07:48.992915 409 raft_consensus.cc:1261] T 9811bfa9f5ca489e953f18fb61cfaad9 P b66124a6ca6244748b38ebb07c6ca40e [term 1 FOLLOWER]: Refusing update from remote peer 7136de9ce16f4703a928d1429952cdea: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:07:48.994179 1096 consensus_queue.cc:1035] T 9811bfa9f5ca489e953f18fb61cfaad9 P 7136de9ce16f4703a928d1429952cdea [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8b65b15973c4d7d896a96567f97caee" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 38009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:07:48.994942 1096 consensus_queue.cc:1035] T 9811bfa9f5ca489e953f18fb61cfaad9 P 7136de9ce16f4703a928d1429952cdea [LEADER]: Connected to new peer: Peer: permanent_uuid: "b66124a6ca6244748b38ebb07c6ca40e" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:07:49.018877 1098 tablet_replica.cc:439] T 9811bfa9f5ca489e953f18fb61cfaad9 P e8b65b15973c4d7d896a96567f97caee: TxnStatusTablet state changed. Reason: New leader 7136de9ce16f4703a928d1429952cdea. Latest consensus state: current_term: 1 leader_uuid: "7136de9ce16f4703a928d1429952cdea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b66124a6ca6244748b38ebb07c6ca40e" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41515 } } peers { permanent_uuid: "e8b65b15973c4d7d896a96567f97caee" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 38009 } } peers { permanent_uuid: "7136de9ce16f4703a928d1429952cdea" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 45057 } } }
I20260811 09:07:49.019214 1098 tablet_replica.cc:442] T 9811bfa9f5ca489e953f18fb61cfaad9 P e8b65b15973c4d7d896a96567f97caee: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 09:07:49.022346 1097 tablet_replica.cc:439] T 9811bfa9f5ca489e953f18fb61cfaad9 P b66124a6ca6244748b38ebb07c6ca40e: TxnStatusTablet state changed. Reason: New leader 7136de9ce16f4703a928d1429952cdea. Latest consensus state: current_term: 1 leader_uuid: "7136de9ce16f4703a928d1429952cdea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b66124a6ca6244748b38ebb07c6ca40e" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41515 } } peers { permanent_uuid: "e8b65b15973c4d7d896a96567f97caee" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 38009 } } peers { permanent_uuid: "7136de9ce16f4703a928d1429952cdea" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 45057 } } }
I20260811 09:07:49.022687 1097 tablet_replica.cc:442] T 9811bfa9f5ca489e953f18fb61cfaad9 P b66124a6ca6244748b38ebb07c6ca40e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 09:07:49.023105 1101 tablet_replica.cc:439] T 9811bfa9f5ca489e953f18fb61cfaad9 P 7136de9ce16f4703a928d1429952cdea: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7136de9ce16f4703a928d1429952cdea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b66124a6ca6244748b38ebb07c6ca40e" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41515 } } peers { permanent_uuid: "e8b65b15973c4d7d896a96567f97caee" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 38009 } } peers { permanent_uuid: "7136de9ce16f4703a928d1429952cdea" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 45057 } } }
I20260811 09:07:49.023388 1101 tablet_replica.cc:442] T 9811bfa9f5ca489e953f18fb61cfaad9 P 7136de9ce16f4703a928d1429952cdea: This TxnStatusTablet replica's current role is: LEADER
I20260811 09:07:49.024021 1143 mvcc.cc:205] Tried to move back new op lower bound from 7317255245782581248 to 7317255245658906624. Current Snapshot: MvccSnapshot[applied={T|T < 7317255245782581248}]
I20260811 09:07:49.024319 1101 tablet_replica.cc:439] T 9811bfa9f5ca489e953f18fb61cfaad9 P 7136de9ce16f4703a928d1429952cdea: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7136de9ce16f4703a928d1429952cdea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b66124a6ca6244748b38ebb07c6ca40e" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41515 } } peers { permanent_uuid: "e8b65b15973c4d7d896a96567f97caee" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 38009 } } peers { permanent_uuid: "7136de9ce16f4703a928d1429952cdea" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 45057 } } }
I20260811 09:07:49.024581 1101 tablet_replica.cc:442] T 9811bfa9f5ca489e953f18fb61cfaad9 P 7136de9ce16f4703a928d1429952cdea: This TxnStatusTablet replica's current role is: LEADER
I20260811 09:07:49.026443 1098 tablet_replica.cc:439] T 9811bfa9f5ca489e953f18fb61cfaad9 P e8b65b15973c4d7d896a96567f97caee: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7136de9ce16f4703a928d1429952cdea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b66124a6ca6244748b38ebb07c6ca40e" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41515 } } peers { permanent_uuid: "e8b65b15973c4d7d896a96567f97caee" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 38009 } } peers { permanent_uuid: "7136de9ce16f4703a928d1429952cdea" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 45057 } } }
I20260811 09:07:49.026713 1098 tablet_replica.cc:442] T 9811bfa9f5ca489e953f18fb61cfaad9 P e8b65b15973c4d7d896a96567f97caee: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 09:07:49.029356 1097 tablet_replica.cc:439] T 9811bfa9f5ca489e953f18fb61cfaad9 P b66124a6ca6244748b38ebb07c6ca40e: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7136de9ce16f4703a928d1429952cdea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b66124a6ca6244748b38ebb07c6ca40e" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41515 } } peers { permanent_uuid: "e8b65b15973c4d7d896a96567f97caee" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 38009 } } peers { permanent_uuid: "7136de9ce16f4703a928d1429952cdea" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 45057 } } }
I20260811 09:07:49.029614 1097 tablet_replica.cc:442] T 9811bfa9f5ca489e953f18fb61cfaad9 P b66124a6ca6244748b38ebb07c6ca40e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 09:07:49.040535 1073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:49.034619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37230 (local address 127.28.154.131:45057)
0811 09:07:49.034787 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:49.034794 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:49.035084 (+ 290us) server_negotiation.cc:407] Connection header received
0811 09:07:49.035266 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:49.035270 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:49.035339 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:49.035411 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:07:49.035810 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:49.036664 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:49.038589 (+ 1925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:49.039018 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:49.039145 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:49.039656 (+ 511us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:07:49.039879 (+ 223us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:07:49.039968 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:49.040134 (+ 166us) server_negotiation.cc:299] Negotiation successful
0811 09:07:49.040204 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 09:07:49.058322 1073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:49.052042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.131:46697 (local address 127.28.154.131:45057)
0811 09:07:49.052218 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:49.052225 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:49.052801 (+ 576us) server_negotiation.cc:407] Connection header received
0811 09:07:49.052957 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:49.052962 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:49.053050 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:49.053155 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:07:49.053624 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:49.054757 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:49.056142 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:49.056873 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:49.056995 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:49.057484 (+ 489us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:07:49.057680 (+ 196us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:07:49.057773 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:49.057915 (+ 142us) server_negotiation.cc:299] Negotiation successful
0811 09:07:49.058002 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 09:07:49.058353 1155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:49.052025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.131:45057 (local address 127.28.154.131:46697)
0811 09:07:49.052705 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:49.052723 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 09:07:49.052846 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:07:49.053171 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:07:49.053174 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:07:49.053184 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 09:07:49.053487 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:49.053494 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:49.054901 (+ 1407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:49.054905 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:49.055998 (+ 1093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:49.056010 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:49.057089 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:49.057094 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:49.057219 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:49.057279 (+ 60us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 09:07:49.057789 (+ 510us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 09:07:49.057799 (+ 10us) client_negotiation.cc:769] Sending connection context
0811 09:07:49.057872 (+ 73us) client_negotiation.cc:240] Negotiation successful
0811 09:07:49.057987 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":510,"thread_start_us":368,"threads_started":1}
I20260811 09:07:49.069470 685 raft_consensus.cc:1261] T 6a96ff9f1a9e4005ace3f929022acf40 P e8b65b15973c4d7d896a96567f97caee [term 1 FOLLOWER]: Refusing update from remote peer 7136de9ce16f4703a928d1429952cdea: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:07:49.069644 409 raft_consensus.cc:1261] T 6a96ff9f1a9e4005ace3f929022acf40 P b66124a6ca6244748b38ebb07c6ca40e [term 1 FOLLOWER]: Refusing update from remote peer 7136de9ce16f4703a928d1429952cdea: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:07:49.070138 1156 consensus_queue.cc:1035] T 6a96ff9f1a9e4005ace3f929022acf40 P 7136de9ce16f4703a928d1429952cdea [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8b65b15973c4d7d896a96567f97caee" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 38009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:07:49.070628 1101 consensus_queue.cc:1035] T 6a96ff9f1a9e4005ace3f929022acf40 P 7136de9ce16f4703a928d1429952cdea [LEADER]: Connected to new peer: Peer: permanent_uuid: "b66124a6ca6244748b38ebb07c6ca40e" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:07:49.075721 1135 mvcc.cc:205] Tried to move back new op lower bound from 7317255246103187456 to 7317255244642811904. Current Snapshot: MvccSnapshot[applied={T|T < 7317255246103187456}]
I20260811 09:07:49.077924 1137 mvcc.cc:205] Tried to move back new op lower bound from 7317255246103187456 to 7317255244642811904. Current Snapshot: MvccSnapshot[applied={T|T < 7317255246103187456}]
I20260811 09:07:49.143154 32563 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47742:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:07:49.143465 32563 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47742:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:07:49.147439 32563 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 49c59055f30d4f5e8fc4a7bfebbdd512: Sending DeleteTablet for 3 replicas of tablet 6a96ff9f1a9e4005ace3f929022acf40
I20260811 09:07:49.148583 908 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a96ff9f1a9e4005ace3f929022acf40 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:07:49 UTC) from {username='kudu', principal='kudu/127.28.154.190@KRBTEST.COM'} at 127.0.0.1:37208
I20260811 09:07:49.149015 644 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a96ff9f1a9e4005ace3f929022acf40 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:07:49 UTC) from {username='kudu', principal='kudu/127.28.154.190@KRBTEST.COM'} at 127.0.0.1:59830
I20260811 09:07:49.149469 361 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a96ff9f1a9e4005ace3f929022acf40 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:07:49 UTC) from {username='kudu', principal='kudu/127.28.154.190@KRBTEST.COM'} at 127.0.0.1:49252
I20260811 09:07:49.150075 32563 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47742:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 09:07:49.150997 32563 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 09:07:49.153609 1172 tablet_replica.cc:330] T 6a96ff9f1a9e4005ace3f929022acf40 P b66124a6ca6244748b38ebb07c6ca40e: stopping tablet replica
I20260811 09:07:49.154232 1172 raft_consensus.cc:2229] T 6a96ff9f1a9e4005ace3f929022acf40 P b66124a6ca6244748b38ebb07c6ca40e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:07:49.154776 1172 raft_consensus.cc:2258] T 6a96ff9f1a9e4005ace3f929022acf40 P b66124a6ca6244748b38ebb07c6ca40e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:07:49.162802 641 tablet_service.cc:1467] Processing CreateTablet for tablet fcb7f72b3f2b484383b6fd8b51bd3e63 (DEFAULT_TABLE table=test-table [id=64c58c9b43474eb5810bed9bdc5481b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:07:49.163586 641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fcb7f72b3f2b484383b6fd8b51bd3e63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:07:49.165002 1171 tablet_replica.cc:330] T 6a96ff9f1a9e4005ace3f929022acf40 P e8b65b15973c4d7d896a96567f97caee: stopping tablet replica
I20260811 09:07:49.165004 1172 ts_tablet_manager.cc:1905] T 6a96ff9f1a9e4005ace3f929022acf40 P b66124a6ca6244748b38ebb07c6ca40e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:07:49.165230 910 tablet_service.cc:1467] Processing CreateTablet for tablet fcb7f72b3f2b484383b6fd8b51bd3e63 (DEFAULT_TABLE table=test-table [id=64c58c9b43474eb5810bed9bdc5481b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:07:49.165578 1171 raft_consensus.cc:2229] T 6a96ff9f1a9e4005ace3f929022acf40 P e8b65b15973c4d7d896a96567f97caee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:07:49.165709 910 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fcb7f72b3f2b484383b6fd8b51bd3e63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:07:49.165932 1171 raft_consensus.cc:2258] T 6a96ff9f1a9e4005ace3f929022acf40 P e8b65b15973c4d7d896a96567f97caee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:07:49.168951 1167 tablet_replica.cc:330] T 6a96ff9f1a9e4005ace3f929022acf40 P 7136de9ce16f4703a928d1429952cdea: stopping tablet replica
I20260811 09:07:49.170588 1167 raft_consensus.cc:2229] T 6a96ff9f1a9e4005ace3f929022acf40 P 7136de9ce16f4703a928d1429952cdea [term 1 LEADER]: Raft consensus shutting down.
I20260811 09:07:49.171113 361 tablet_service.cc:1467] Processing CreateTablet for tablet fcb7f72b3f2b484383b6fd8b51bd3e63 (DEFAULT_TABLE table=test-table [id=64c58c9b43474eb5810bed9bdc5481b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:07:49.171306 1167 raft_consensus.cc:2258] T 6a96ff9f1a9e4005ace3f929022acf40 P 7136de9ce16f4703a928d1429952cdea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:07:49.171547 361 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fcb7f72b3f2b484383b6fd8b51bd3e63. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:07:49.173249 1167 ts_tablet_manager.cc:1905] T 6a96ff9f1a9e4005ace3f929022acf40 P 7136de9ce16f4703a928d1429952cdea: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:07:49.175140 1172 ts_tablet_manager.cc:1918] T 6a96ff9f1a9e4005ace3f929022acf40 P b66124a6ca6244748b38ebb07c6ca40e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 09:07:49.175295 1172 log.cc:1196] T 6a96ff9f1a9e4005ace3f929022acf40 P b66124a6ca6244748b38ebb07c6ca40e: Deleting WAL directory at /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439250192320-29290-0/minicluster-data/ts-0/wal/wals/6a96ff9f1a9e4005ace3f929022acf40
I20260811 09:07:49.175868 1172 ts_tablet_manager.cc:1939] T 6a96ff9f1a9e4005ace3f929022acf40 P b66124a6ca6244748b38ebb07c6ca40e: Deleting consensus metadata
I20260811 09:07:49.178004 32534 catalog_manager.cc:4841] TS b66124a6ca6244748b38ebb07c6ca40e (127.28.154.129:41515): tablet 6a96ff9f1a9e4005ace3f929022acf40 (table test-table [id=2a3cf6dca65843dc9d483a8007c19471]) successfully deleted
I20260811 09:07:49.182528 1083 tablet_bootstrap.cc:492] T fcb7f72b3f2b484383b6fd8b51bd3e63 P 7136de9ce16f4703a928d1429952cdea: Bootstrap starting.
I20260811 09:07:49.183947 1167 ts_tablet_manager.cc:1918] T 6a96ff9f1a9e4005ace3f929022acf40 P 7136de9ce16f4703a928d1429952cdea: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 09:07:49.184134 1167 log.cc:1196] T 6a96ff9f1a9e4005ace3f929022acf40 P 7136de9ce16f4703a928d1429952cdea: Deleting WAL directory at /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439250192320-29290-0/minicluster-data/ts-2/wal/wals/6a96ff9f1a9e4005ace3f929022acf40
I20260811 09:07:49.184377 1083 tablet_bootstrap.cc:654] T fcb7f72b3f2b484383b6fd8b51bd3e63 P 7136de9ce16f4703a928d1429952cdea: Neither blocks nor log segments found. Creating new log.
I20260811 09:07:49.185503 1082 tablet_bootstrap.cc:492] T fcb7f72b3f2b484383b6fd8b51bd3e63 P e8b65b15973c4d7d896a96567f97caee: Bootstrap starting.
I20260811 09:07:49.185992 1167 ts_tablet_manager.cc:1939] T 6a96ff9f1a9e4005ace3f929022acf40 P 7136de9ce16f4703a928d1429952cdea: Deleting consensus metadata
I20260811 09:07:49.186491 1171 ts_tablet_manager.cc:1905] T 6a96ff9f1a9e4005ace3f929022acf40 P e8b65b15973c4d7d896a96567f97caee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:07:49.187175 1082 tablet_bootstrap.cc:654] T fcb7f72b3f2b484383b6fd8b51bd3e63 P e8b65b15973c4d7d896a96567f97caee: Neither blocks nor log segments found. Creating new log.
I20260811 09:07:49.188978 32534 catalog_manager.cc:4841] TS 7136de9ce16f4703a928d1429952cdea (127.28.154.131:45057): tablet 6a96ff9f1a9e4005ace3f929022acf40 (table test-table [id=2a3cf6dca65843dc9d483a8007c19471]) successfully deleted
I20260811 09:07:49.193179 1085 tablet_bootstrap.cc:492] T fcb7f72b3f2b484383b6fd8b51bd3e63 P b66124a6ca6244748b38ebb07c6ca40e: Bootstrap starting.
I20260811 09:07:49.193638 1082 tablet_bootstrap.cc:492] T fcb7f72b3f2b484383b6fd8b51bd3e63 P e8b65b15973c4d7d896a96567f97caee: No bootstrap required, opened a new log
I20260811 09:07:49.193796 1082 ts_tablet_manager.cc:1397] T fcb7f72b3f2b484383b6fd8b51bd3e63 P e8b65b15973c4d7d896a96567f97caee: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.000s
I20260811 09:07:49.194257 1171 ts_tablet_manager.cc:1918] T 6a96ff9f1a9e4005ace3f929022acf40 P e8b65b15973c4d7d896a96567f97caee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 09:07:49.194451 1171 log.cc:1196] T 6a96ff9f1a9e4005ace3f929022acf40 P e8b65b15973c4d7d896a96567f97caee: Deleting WAL directory at /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439250192320-29290-0/minicluster-data/ts-1/wal/wals/6a96ff9f1a9e4005ace3f929022acf40
I20260811 09:07:49.194936 1085 tablet_bootstrap.cc:654] T fcb7f72b3f2b484383b6fd8b51bd3e63 P b66124a6ca6244748b38ebb07c6ca40e: Neither blocks nor log segments found. Creating new log.
I20260811 09:07:49.194866 1082 raft_consensus.cc:348] T fcb7f72b3f2b484383b6fd8b51bd3e63 P e8b65b15973c4d7d896a96567f97caee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b66124a6ca6244748b38ebb07c6ca40e" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41515 } } peers { permanent_uuid: "e8b65b15973c4d7d896a96567f97caee" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 38009 } } peers { permanent_uuid: "7136de9ce16f4703a928d1429952cdea" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 45057 } }
I20260811 09:07:49.195302 1082 raft_consensus.cc:374] T fcb7f72b3f2b484383b6fd8b51bd3e63 P e8b65b15973c4d7d896a96567f97caee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:07:49.195374 1082 raft_consensus.cc:729] T fcb7f72b3f2b484383b6fd8b51bd3e63 P e8b65b15973c4d7d896a96567f97caee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8b65b15973c4d7d896a96567f97caee, State: Initialized, Role: FOLLOWER
I20260811 09:07:49.195696 1082 consensus_queue.cc:260] T fcb7f72b3f2b484383b6fd8b51bd3e63 P e8b65b15973c4d7d896a96567f97caee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b66124a6ca6244748b38ebb07c6ca40e" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41515 } } peers { permanent_uuid: "e8b65b15973c4d7d896a96567f97caee" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 38009 } } peers { permanent_uuid: "7136de9ce16f4703a928d1429952cdea" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 45057 } }
I20260811 09:07:49.199035 1082 ts_tablet_manager.cc:1428] T fcb7f72b3f2b484383b6fd8b51bd3e63 P e8b65b15973c4d7d896a96567f97caee: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260811 09:07:49.199113 1083 tablet_bootstrap.cc:492] T fcb7f72b3f2b484383b6fd8b51bd3e63 P 7136de9ce16f4703a928d1429952cdea: No bootstrap required, opened a new log
I20260811 09:07:49.199306 1083 ts_tablet_manager.cc:1397] T fcb7f72b3f2b484383b6fd8b51bd3e63 P 7136de9ce16f4703a928d1429952cdea: Time spent bootstrapping tablet: real 0.017s user 0.000s sys 0.009s
I20260811 09:07:49.200448 1083 raft_consensus.cc:348] T fcb7f72b3f2b484383b6fd8b51bd3e63 P 7136de9ce16f4703a928d1429952cdea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b66124a6ca6244748b38ebb07c6ca40e" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41515 } } peers { permanent_uuid: "e8b65b15973c4d7d896a96567f97caee" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 38009 } } peers { permanent_uuid: "7136de9ce16f4703a928d1429952cdea" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 45057 } }
I20260811 09:07:49.200695 1083 raft_consensus.cc:374] T fcb7f72b3f2b484383b6fd8b51bd3e63 P 7136de9ce16f4703a928d1429952cdea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:07:49.200783 1083 raft_consensus.cc:729] T fcb7f72b3f2b484383b6fd8b51bd3e63 P 7136de9ce16f4703a928d1429952cdea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7136de9ce16f4703a928d1429952cdea, State: Initialized, Role: FOLLOWER
I20260811 09:07:49.201030 1083 consensus_queue.cc:260] T fcb7f72b3f2b484383b6fd8b51bd3e63 P 7136de9ce16f4703a928d1429952cdea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b66124a6ca6244748b38ebb07c6ca40e" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41515 } } peers { permanent_uuid: "e8b65b15973c4d7d896a96567f97caee" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 38009 } } peers { permanent_uuid: "7136de9ce16f4703a928d1429952cdea" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 45057 } }
I20260811 09:07:49.201731 1083 ts_tablet_manager.cc:1428] T fcb7f72b3f2b484383b6fd8b51bd3e63 P 7136de9ce16f4703a928d1429952cdea: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 09:07:49.194988 1171 ts_tablet_manager.cc:1939] T 6a96ff9f1a9e4005ace3f929022acf40 P e8b65b15973c4d7d896a96567f97caee: Deleting consensus metadata
I20260811 09:07:49.204018 32534 catalog_manager.cc:4841] TS e8b65b15973c4d7d896a96567f97caee (127.28.154.130:38009): tablet 6a96ff9f1a9e4005ace3f929022acf40 (table test-table [id=2a3cf6dca65843dc9d483a8007c19471]) successfully deleted
I20260811 09:07:49.205127 1085 tablet_bootstrap.cc:492] T fcb7f72b3f2b484383b6fd8b51bd3e63 P b66124a6ca6244748b38ebb07c6ca40e: No bootstrap required, opened a new log
I20260811 09:07:49.205351 1085 ts_tablet_manager.cc:1397] T fcb7f72b3f2b484383b6fd8b51bd3e63 P b66124a6ca6244748b38ebb07c6ca40e: Time spent bootstrapping tablet: real 0.012s user 0.000s sys 0.008s
I20260811 09:07:49.206000 1085 raft_consensus.cc:348] T fcb7f72b3f2b484383b6fd8b51bd3e63 P b66124a6ca6244748b38ebb07c6ca40e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b66124a6ca6244748b38ebb07c6ca40e" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41515 } } peers { permanent_uuid: "e8b65b15973c4d7d896a96567f97caee" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 38009 } } peers { permanent_uuid: "7136de9ce16f4703a928d1429952cdea" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 45057 } }
I20260811 09:07:49.206200 1085 raft_consensus.cc:374] T fcb7f72b3f2b484383b6fd8b51bd3e63 P b66124a6ca6244748b38ebb07c6ca40e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:07:49.206274 1085 raft_consensus.cc:729] T fcb7f72b3f2b484383b6fd8b51bd3e63 P b66124a6ca6244748b38ebb07c6ca40e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b66124a6ca6244748b38ebb07c6ca40e, State: Initialized, Role: FOLLOWER
I20260811 09:07:49.206519 1085 consensus_queue.cc:260] T fcb7f72b3f2b484383b6fd8b51bd3e63 P b66124a6ca6244748b38ebb07c6ca40e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b66124a6ca6244748b38ebb07c6ca40e" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41515 } } peers { permanent_uuid: "e8b65b15973c4d7d896a96567f97caee" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 38009 } } peers { permanent_uuid: "7136de9ce16f4703a928d1429952cdea" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 45057 } }
I20260811 09:07:49.207024 1085 ts_tablet_manager.cc:1428] T fcb7f72b3f2b484383b6fd8b51bd3e63 P b66124a6ca6244748b38ebb07c6ca40e: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260811 09:07:49.219672 1098 raft_consensus.cc:482] T fcb7f72b3f2b484383b6fd8b51bd3e63 P e8b65b15973c4d7d896a96567f97caee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:07:49.219923 1098 raft_consensus.cc:504] T fcb7f72b3f2b484383b6fd8b51bd3e63 P e8b65b15973c4d7d896a96567f97caee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b66124a6ca6244748b38ebb07c6ca40e" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41515 } } peers { permanent_uuid: "e8b65b15973c4d7d896a96567f97caee" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 38009 } } peers { permanent_uuid: "7136de9ce16f4703a928d1429952cdea" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 45057 } }
I20260811 09:07:49.221478 1098 leader_election.cc:288] T fcb7f72b3f2b484383b6fd8b51bd3e63 P e8b65b15973c4d7d896a96567f97caee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b66124a6ca6244748b38ebb07c6ca40e (127.28.154.129:41515), 7136de9ce16f4703a928d1429952cdea (127.28.154.131:45057)
I20260811 09:07:49.230458 1073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:49.222798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.130:41977 (local address 127.28.154.131:45057)
0811 09:07:49.222982 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:49.222988 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:49.223543 (+ 555us) server_negotiation.cc:407] Connection header received
0811 09:07:49.223832 (+ 289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:49.223839 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:49.223932 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:49.224056 (+ 124us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:07:49.225009 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:49.226072 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:49.228069 (+ 1997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:49.229379 (+ 1310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:49.229557 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:49.229634 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:49.229967 (+ 333us) server_negotiation.cc:299] Negotiation successful
0811 09:07:49.230127 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 09:07:49.230744 1176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:49.221752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.129:41515 (local address 127.28.154.130:44793)
0811 09:07:49.222253 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:49.222273 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 09:07:49.222417 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:07:49.223984 (+ 1567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:07:49.223988 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:07:49.224015 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:07:49.224305 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:49.224318 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:49.225922 (+ 1604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:49.225928 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:49.228313 (+ 2385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:49.228330 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:49.229916 (+ 1586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:49.229922 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:49.230071 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:49.230095 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 09:07:49.230176 (+ 81us) client_negotiation.cc:240] Negotiation successful
0811 09:07:49.230307 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":374,"mutex_wait_us":24,"thread_start_us":268,"threads_started":1}
I20260811 09:07:49.231166 1177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:49.221850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.130:44793 (local address 127.28.154.129:41515)
0811 09:07:49.223721 (+ 1871us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:49.223727 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:49.223743 (+ 16us) server_negotiation.cc:407] Connection header received
0811 09:07:49.223809 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:49.223813 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:49.223867 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:49.223951 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:07:49.224561 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:49.225778 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:49.228494 (+ 2716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:49.229700 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:49.229852 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:49.229931 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:49.230219 (+ 288us) server_negotiation.cc:299] Negotiation successful
0811 09:07:49.230357 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1746,"thread_start_us":1597,"threads_started":1}
I20260811 09:07:49.231235 1180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:49.222429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.131:45057 (local address 127.28.154.130:41977)
0811 09:07:49.223447 (+ 1018us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:49.223464 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 09:07:49.223651 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:07:49.224437 (+ 786us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:07:49.224442 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:07:49.224468 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:07:49.224831 (+ 363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:49.224844 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:49.226236 (+ 1392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:49.226242 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:49.227934 (+ 1692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:49.227952 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:49.229558 (+ 1606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:49.229564 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:49.229745 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:49.229783 (+ 38us) client_negotiation.cc:769] Sending connection context
0811 09:07:49.230670 (+ 887us) client_negotiation.cc:240] Negotiation successful
0811 09:07:49.230822 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":889,"mutex_wait_us":7,"thread_start_us":721,"threads_started":1}
I20260811 09:07:49.231767 409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcb7f72b3f2b484383b6fd8b51bd3e63" candidate_uuid: "e8b65b15973c4d7d896a96567f97caee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b66124a6ca6244748b38ebb07c6ca40e" is_pre_election: true
I20260811 09:07:49.232048 409 raft_consensus.cc:2454] T fcb7f72b3f2b484383b6fd8b51bd3e63 P b66124a6ca6244748b38ebb07c6ca40e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8b65b15973c4d7d896a96567f97caee in term 0.
I20260811 09:07:49.232201 952 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcb7f72b3f2b484383b6fd8b51bd3e63" candidate_uuid: "e8b65b15973c4d7d896a96567f97caee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7136de9ce16f4703a928d1429952cdea" is_pre_election: true
I20260811 09:07:49.232626 540 leader_election.cc:302] T fcb7f72b3f2b484383b6fd8b51bd3e63 P e8b65b15973c4d7d896a96567f97caee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b66124a6ca6244748b38ebb07c6ca40e, e8b65b15973c4d7d896a96567f97caee; no voters:
I20260811 09:07:49.232715 952 raft_consensus.cc:2454] T fcb7f72b3f2b484383b6fd8b51bd3e63 P 7136de9ce16f4703a928d1429952cdea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8b65b15973c4d7d896a96567f97caee in term 0.
I20260811 09:07:49.233104 1098 raft_consensus.cc:2781] T fcb7f72b3f2b484383b6fd8b51bd3e63 P e8b65b15973c4d7d896a96567f97caee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 09:07:49.233243 1098 raft_consensus.cc:482] T fcb7f72b3f2b484383b6fd8b51bd3e63 P e8b65b15973c4d7d896a96567f97caee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 09:07:49.233342 1098 raft_consensus.cc:3037] T fcb7f72b3f2b484383b6fd8b51bd3e63 P e8b65b15973c4d7d896a96567f97caee [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:07:49.236639 1098 raft_consensus.cc:504] T fcb7f72b3f2b484383b6fd8b51bd3e63 P e8b65b15973c4d7d896a96567f97caee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b66124a6ca6244748b38ebb07c6ca40e" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41515 } } peers { permanent_uuid: "e8b65b15973c4d7d896a96567f97caee" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 38009 } } peers { permanent_uuid: "7136de9ce16f4703a928d1429952cdea" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 45057 } }
I20260811 09:07:49.237294 1098 leader_election.cc:288] T fcb7f72b3f2b484383b6fd8b51bd3e63 P e8b65b15973c4d7d896a96567f97caee [CANDIDATE]: Term 1 election: Requested vote from peers b66124a6ca6244748b38ebb07c6ca40e (127.28.154.129:41515), 7136de9ce16f4703a928d1429952cdea (127.28.154.131:45057)
I20260811 09:07:49.237926 409 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcb7f72b3f2b484383b6fd8b51bd3e63" candidate_uuid: "e8b65b15973c4d7d896a96567f97caee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b66124a6ca6244748b38ebb07c6ca40e"
I20260811 09:07:49.238124 409 raft_consensus.cc:3037] T fcb7f72b3f2b484383b6fd8b51bd3e63 P b66124a6ca6244748b38ebb07c6ca40e [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:07:49.240211 409 raft_consensus.cc:2454] T fcb7f72b3f2b484383b6fd8b51bd3e63 P b66124a6ca6244748b38ebb07c6ca40e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8b65b15973c4d7d896a96567f97caee in term 1.
I20260811 09:07:49.240711 952 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcb7f72b3f2b484383b6fd8b51bd3e63" candidate_uuid: "e8b65b15973c4d7d896a96567f97caee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7136de9ce16f4703a928d1429952cdea"
I20260811 09:07:49.240784 540 leader_election.cc:302] T fcb7f72b3f2b484383b6fd8b51bd3e63 P e8b65b15973c4d7d896a96567f97caee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b66124a6ca6244748b38ebb07c6ca40e, e8b65b15973c4d7d896a96567f97caee; no voters:
I20260811 09:07:49.240948 952 raft_consensus.cc:3037] T fcb7f72b3f2b484383b6fd8b51bd3e63 P 7136de9ce16f4703a928d1429952cdea [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:07:49.241284 1098 raft_consensus.cc:2781] T fcb7f72b3f2b484383b6fd8b51bd3e63 P e8b65b15973c4d7d896a96567f97caee [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:07:49.243323 952 raft_consensus.cc:2454] T fcb7f72b3f2b484383b6fd8b51bd3e63 P 7136de9ce16f4703a928d1429952cdea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8b65b15973c4d7d896a96567f97caee in term 1.
I20260811 09:07:49.243580 1098 raft_consensus.cc:686] T fcb7f72b3f2b484383b6fd8b51bd3e63 P e8b65b15973c4d7d896a96567f97caee [term 1 LEADER]: Becoming Leader. State: Replica: e8b65b15973c4d7d896a96567f97caee, State: Running, Role: LEADER
I20260811 09:07:49.244053 1098 consensus_queue.cc:237] T fcb7f72b3f2b484383b6fd8b51bd3e63 P e8b65b15973c4d7d896a96567f97caee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b66124a6ca6244748b38ebb07c6ca40e" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41515 } } peers { permanent_uuid: "e8b65b15973c4d7d896a96567f97caee" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 38009 } } peers { permanent_uuid: "7136de9ce16f4703a928d1429952cdea" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 45057 } }
I20260811 09:07:49.248157 32567 catalog_manager.cc:5495] T fcb7f72b3f2b484383b6fd8b51bd3e63 P e8b65b15973c4d7d896a96567f97caee reported cstate change: term changed from 0 to 1, leader changed from <none> to e8b65b15973c4d7d896a96567f97caee (127.28.154.130). New cstate: current_term: 1 leader_uuid: "e8b65b15973c4d7d896a96567f97caee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b66124a6ca6244748b38ebb07c6ca40e" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8b65b15973c4d7d896a96567f97caee" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 38009 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7136de9ce16f4703a928d1429952cdea" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 45057 } health_report { overall_health: UNKNOWN } } }
I20260811 09:07:49.282414 1188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:49.275675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59832 (local address 127.28.154.130:38009)
0811 09:07:49.276268 (+ 593us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:49.276272 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:49.276291 (+ 19us) server_negotiation.cc:407] Connection header received
0811 09:07:49.276414 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:49.276418 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:49.276502 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:49.276663 (+ 161us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:07:49.277077 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:49.277867 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:49.279238 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:49.280097 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:49.280310 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:49.281057 (+ 747us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:07:49.281721 (+ 664us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:07:49.281859 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:49.281971 (+ 112us) server_negotiation.cc:299] Negotiation successful
0811 09:07:49.282069 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":469,"thread_start_us":314,"threads_started":1}
I20260811 09:07:49.301478 1073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:49.294420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.130:48631 (local address 127.28.154.131:45057)
0811 09:07:49.294836 (+ 416us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:49.294843 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:49.295250 (+ 407us) server_negotiation.cc:407] Connection header received
0811 09:07:49.295439 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:49.295445 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:49.295537 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:49.295655 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:07:49.296114 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:49.297311 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:49.299110 (+ 1799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:49.299740 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:49.299903 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:49.300669 (+ 766us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:07:49.300820 (+ 151us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:07:49.300922 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:49.301083 (+ 161us) server_negotiation.cc:299] Negotiation successful
0811 09:07:49.301177 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260811 09:07:49.301501 1192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:49.294379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.131:45057 (local address 127.28.154.130:48631)
0811 09:07:49.295138 (+ 759us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:49.295155 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 09:07:49.295304 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:07:49.295664 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:07:49.295668 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:07:49.295677 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 09:07:49.295961 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:49.295969 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:49.297492 (+ 1523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:49.297496 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:49.298941 (+ 1445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:49.298958 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:49.299969 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:49.299975 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:49.300327 (+ 352us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:49.300458 (+ 131us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 09:07:49.300947 (+ 489us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 09:07:49.300960 (+ 13us) client_negotiation.cc:769] Sending connection context
0811 09:07:49.301043 (+ 83us) client_negotiation.cc:240] Negotiation successful
0811 09:07:49.301177 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":554,"thread_start_us":400,"threads_started":1}
I20260811 09:07:49.312059 409 raft_consensus.cc:1261] T fcb7f72b3f2b484383b6fd8b51bd3e63 P b66124a6ca6244748b38ebb07c6ca40e [term 1 FOLLOWER]: Refusing update from remote peer e8b65b15973c4d7d896a96567f97caee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:07:49.312229 952 raft_consensus.cc:1261] T fcb7f72b3f2b484383b6fd8b51bd3e63 P 7136de9ce16f4703a928d1429952cdea [term 1 FOLLOWER]: Refusing update from remote peer e8b65b15973c4d7d896a96567f97caee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:07:49.312963 1185 consensus_queue.cc:1035] T fcb7f72b3f2b484383b6fd8b51bd3e63 P e8b65b15973c4d7d896a96567f97caee [LEADER]: Connected to new peer: Peer: permanent_uuid: "b66124a6ca6244748b38ebb07c6ca40e" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:07:49.313318 1098 consensus_queue.cc:1035] T fcb7f72b3f2b484383b6fd8b51bd3e63 P e8b65b15973c4d7d896a96567f97caee [LEADER]: Connected to new peer: Peer: permanent_uuid: "7136de9ce16f4703a928d1429952cdea" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 45057 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:07:49.348362 1155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:49.341557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.130:38009 (local address 127.28.154.131:36719)
0811 09:07:49.341778 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:49.341802 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 09:07:49.341922 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:07:49.342614 (+ 692us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:07:49.342623 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:07:49.342635 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 09:07:49.342847 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:49.342865 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:49.344329 (+ 1464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:49.344364 (+ 35us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:49.345628 (+ 1264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:49.345643 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:49.346526 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:49.346533 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:49.346768 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:49.346824 (+ 56us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 09:07:49.347731 (+ 907us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 09:07:49.347744 (+ 13us) client_negotiation.cc:769] Sending connection context
0811 09:07:49.347838 (+ 94us) client_negotiation.cc:240] Negotiation successful
0811 09:07:49.347994 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260811 09:07:49.348829 1188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:49.342171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.131:36719 (local address 127.28.154.130:38009)
0811 09:07:49.342331 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:49.342336 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:49.342351 (+ 15us) server_negotiation.cc:407] Connection header received
0811 09:07:49.342421 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:49.342425 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:49.342486 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:49.342574 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:07:49.343028 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:49.344147 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:49.345770 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:49.346363 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:49.346497 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:49.346985 (+ 488us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:07:49.347219 (+ 234us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:07:49.347387 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:49.348473 (+ 1086us) server_negotiation.cc:299] Negotiation successful
0811 09:07:49.348559 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
W20260811 09:07:49.349974 1143 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260811 09:07:50.957760 32566 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47742:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:07:50.958072 32566 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47742:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:07:50.962442 32566 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 49c59055f30d4f5e8fc4a7bfebbdd512: Sending DeleteTablet for 3 replicas of tablet fcb7f72b3f2b484383b6fd8b51bd3e63
I20260811 09:07:50.963341 641 tablet_service.cc:1514] Processing DeleteTablet for tablet fcb7f72b3f2b484383b6fd8b51bd3e63 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:07:50 UTC) from {username='kudu', principal='kudu/127.28.154.190@KRBTEST.COM'} at 127.0.0.1:59830
I20260811 09:07:50.963313 361 tablet_service.cc:1514] Processing DeleteTablet for tablet fcb7f72b3f2b484383b6fd8b51bd3e63 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:07:50 UTC) from {username='kudu', principal='kudu/127.28.154.190@KRBTEST.COM'} at 127.0.0.1:49252
I20260811 09:07:50.963464 910 tablet_service.cc:1514] Processing DeleteTablet for tablet fcb7f72b3f2b484383b6fd8b51bd3e63 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:07:50 UTC) from {username='kudu', principal='kudu/127.28.154.190@KRBTEST.COM'} at 127.0.0.1:37208
I20260811 09:07:50.975967 1245 tablet_replica.cc:330] T fcb7f72b3f2b484383b6fd8b51bd3e63 P e8b65b15973c4d7d896a96567f97caee: stopping tablet replica
I20260811 09:07:50.976444 1245 raft_consensus.cc:2229] T fcb7f72b3f2b484383b6fd8b51bd3e63 P e8b65b15973c4d7d896a96567f97caee [term 1 LEADER]: Raft consensus shutting down.
I20260811 09:07:50.977203 1245 raft_consensus.cc:2258] T fcb7f72b3f2b484383b6fd8b51bd3e63 P e8b65b15973c4d7d896a96567f97caee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:07:50.978453 1245 ts_tablet_manager.cc:1905] T fcb7f72b3f2b484383b6fd8b51bd3e63 P e8b65b15973c4d7d896a96567f97caee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:07:50.978590 1246 tablet_replica.cc:330] T fcb7f72b3f2b484383b6fd8b51bd3e63 P b66124a6ca6244748b38ebb07c6ca40e: stopping tablet replica
I20260811 09:07:50.978883 1246 raft_consensus.cc:2229] T fcb7f72b3f2b484383b6fd8b51bd3e63 P b66124a6ca6244748b38ebb07c6ca40e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:07:50.979179 1246 raft_consensus.cc:2258] T fcb7f72b3f2b484383b6fd8b51bd3e63 P b66124a6ca6244748b38ebb07c6ca40e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:07:50.980106 1249 tablet_replica.cc:330] T fcb7f72b3f2b484383b6fd8b51bd3e63 P 7136de9ce16f4703a928d1429952cdea: stopping tablet replica
I20260811 09:07:50.980154 1246 ts_tablet_manager.cc:1905] T fcb7f72b3f2b484383b6fd8b51bd3e63 P b66124a6ca6244748b38ebb07c6ca40e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:07:50.980389 1249 raft_consensus.cc:2229] T fcb7f72b3f2b484383b6fd8b51bd3e63 P 7136de9ce16f4703a928d1429952cdea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:07:50.980710 1249 raft_consensus.cc:2258] T fcb7f72b3f2b484383b6fd8b51bd3e63 P 7136de9ce16f4703a928d1429952cdea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:07:50.981760 1249 ts_tablet_manager.cc:1905] T fcb7f72b3f2b484383b6fd8b51bd3e63 P 7136de9ce16f4703a928d1429952cdea: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:07:50.985854 1245 ts_tablet_manager.cc:1918] T fcb7f72b3f2b484383b6fd8b51bd3e63 P e8b65b15973c4d7d896a96567f97caee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 09:07:50.986068 1245 log.cc:1196] T fcb7f72b3f2b484383b6fd8b51bd3e63 P e8b65b15973c4d7d896a96567f97caee: Deleting WAL directory at /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439250192320-29290-0/minicluster-data/ts-1/wal/wals/fcb7f72b3f2b484383b6fd8b51bd3e63
I20260811 09:07:50.986573 1245 ts_tablet_manager.cc:1939] T fcb7f72b3f2b484383b6fd8b51bd3e63 P e8b65b15973c4d7d896a96567f97caee: Deleting consensus metadata
I20260811 09:07:50.986771 1246 ts_tablet_manager.cc:1918] T fcb7f72b3f2b484383b6fd8b51bd3e63 P b66124a6ca6244748b38ebb07c6ca40e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 09:07:50.987011 1246 log.cc:1196] T fcb7f72b3f2b484383b6fd8b51bd3e63 P b66124a6ca6244748b38ebb07c6ca40e: Deleting WAL directory at /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439250192320-29290-0/minicluster-data/ts-0/wal/wals/fcb7f72b3f2b484383b6fd8b51bd3e63
I20260811 09:07:50.987509 1246 ts_tablet_manager.cc:1939] T fcb7f72b3f2b484383b6fd8b51bd3e63 P b66124a6ca6244748b38ebb07c6ca40e: Deleting consensus metadata
I20260811 09:07:50.987938 32534 catalog_manager.cc:4841] TS e8b65b15973c4d7d896a96567f97caee (127.28.154.130:38009): tablet fcb7f72b3f2b484383b6fd8b51bd3e63 (table test-table [id=64c58c9b43474eb5810bed9bdc5481b9]) successfully deleted
I20260811 09:07:50.988307 1249 ts_tablet_manager.cc:1918] T fcb7f72b3f2b484383b6fd8b51bd3e63 P 7136de9ce16f4703a928d1429952cdea: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 09:07:50.988508 1249 log.cc:1196] T fcb7f72b3f2b484383b6fd8b51bd3e63 P 7136de9ce16f4703a928d1429952cdea: Deleting WAL directory at /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439250192320-29290-0/minicluster-data/ts-2/wal/wals/fcb7f72b3f2b484383b6fd8b51bd3e63
I20260811 09:07:50.988788 32534 catalog_manager.cc:4841] TS b66124a6ca6244748b38ebb07c6ca40e (127.28.154.129:41515): tablet fcb7f72b3f2b484383b6fd8b51bd3e63 (table test-table [id=64c58c9b43474eb5810bed9bdc5481b9]) successfully deleted
I20260811 09:07:50.989221 1249 ts_tablet_manager.cc:1939] T fcb7f72b3f2b484383b6fd8b51bd3e63 P 7136de9ce16f4703a928d1429952cdea: Deleting consensus metadata
I20260811 09:07:50.990159 32534 catalog_manager.cc:4841] TS 7136de9ce16f4703a928d1429952cdea (127.28.154.131:45057): tablet fcb7f72b3f2b484383b6fd8b51bd3e63 (table test-table [id=64c58c9b43474eb5810bed9bdc5481b9]) successfully deleted
Aug 11 09:07:51 dist-test-slave-xt4k krb5kdc[32478](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439271, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 09:07:51 dist-test-slave-xt4k krb5kdc[32478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439271, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.28.154.190@KRBTEST.COM
I20260811 09:07:51.040442 1268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:51.019183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47770 (local address 127.28.154.190:35229)
0811 09:07:51.019776 (+ 593us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:51.019781 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:51.019945 (+ 164us) server_negotiation.cc:407] Connection header received
0811 09:07:51.020197 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:51.020201 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:51.020277 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:51.020437 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:51.023302 (+ 2865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:51.025551 (+ 2249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:51.027435 (+ 1884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:51.028986 (+ 1551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:51.029125 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:51.033971 (+ 4846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:07:51.033978 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:07:51.033984 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:07:51.034039 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:07:51.037340 (+ 3301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:51.038066 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:51.038073 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:51.038077 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:51.038211 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:51.038600 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:51.038605 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:51.038607 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:51.039063 (+ 456us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:07:51.039329 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:51.039755 (+ 426us) server_negotiation.cc:299] Negotiation successful
0811 09:07:51.040019 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":476,"thread_start_us":319,"threads_started":1}
I20260811 09:07:51.044960 32566 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47770:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 09:07:51.045715 32566 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 09:07:51.053469 641 tablet_service.cc:1467] Processing CreateTablet for tablet 5497ddad181d48a0ae9700fbf795c943 (DEFAULT_TABLE table=test-table [id=9800071265fa4bee937aaf52c659028c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:07:51.053498 361 tablet_service.cc:1467] Processing CreateTablet for tablet 5497ddad181d48a0ae9700fbf795c943 (DEFAULT_TABLE table=test-table [id=9800071265fa4bee937aaf52c659028c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:07:51.054019 641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5497ddad181d48a0ae9700fbf795c943. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:07:51.054060 361 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5497ddad181d48a0ae9700fbf795c943. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:07:51.059661 1269 tablet_bootstrap.cc:492] T 5497ddad181d48a0ae9700fbf795c943 P e8b65b15973c4d7d896a96567f97caee: Bootstrap starting.
I20260811 09:07:51.060949 1271 tablet_bootstrap.cc:492] T 5497ddad181d48a0ae9700fbf795c943 P b66124a6ca6244748b38ebb07c6ca40e: Bootstrap starting.
I20260811 09:07:51.060971 910 tablet_service.cc:1467] Processing CreateTablet for tablet 5497ddad181d48a0ae9700fbf795c943 (DEFAULT_TABLE table=test-table [id=9800071265fa4bee937aaf52c659028c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:07:51.061488 910 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5497ddad181d48a0ae9700fbf795c943. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:07:51.061915 1269 tablet_bootstrap.cc:654] T 5497ddad181d48a0ae9700fbf795c943 P e8b65b15973c4d7d896a96567f97caee: Neither blocks nor log segments found. Creating new log.
I20260811 09:07:51.063242 1271 tablet_bootstrap.cc:654] T 5497ddad181d48a0ae9700fbf795c943 P b66124a6ca6244748b38ebb07c6ca40e: Neither blocks nor log segments found. Creating new log.
I20260811 09:07:51.071297 1275 tablet_bootstrap.cc:492] T 5497ddad181d48a0ae9700fbf795c943 P 7136de9ce16f4703a928d1429952cdea: Bootstrap starting.
I20260811 09:07:51.073987 1275 tablet_bootstrap.cc:654] T 5497ddad181d48a0ae9700fbf795c943 P 7136de9ce16f4703a928d1429952cdea: Neither blocks nor log segments found. Creating new log.
I20260811 09:07:51.077560 1269 tablet_bootstrap.cc:492] T 5497ddad181d48a0ae9700fbf795c943 P e8b65b15973c4d7d896a96567f97caee: No bootstrap required, opened a new log
I20260811 09:07:51.077804 1269 ts_tablet_manager.cc:1397] T 5497ddad181d48a0ae9700fbf795c943 P e8b65b15973c4d7d896a96567f97caee: Time spent bootstrapping tablet: real 0.018s user 0.003s sys 0.010s
I20260811 09:07:51.078827 1269 raft_consensus.cc:348] T 5497ddad181d48a0ae9700fbf795c943 P e8b65b15973c4d7d896a96567f97caee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b66124a6ca6244748b38ebb07c6ca40e" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41515 } } peers { permanent_uuid: "e8b65b15973c4d7d896a96567f97caee" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 38009 } } peers { permanent_uuid: "7136de9ce16f4703a928d1429952cdea" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 45057 } }
I20260811 09:07:51.079314 1269 raft_consensus.cc:374] T 5497ddad181d48a0ae9700fbf795c943 P e8b65b15973c4d7d896a96567f97caee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:07:51.079466 1269 raft_consensus.cc:729] T 5497ddad181d48a0ae9700fbf795c943 P e8b65b15973c4d7d896a96567f97caee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8b65b15973c4d7d896a96567f97caee, State: Initialized, Role: FOLLOWER
I20260811 09:07:51.079887 1269 consensus_queue.cc:260] T 5497ddad181d48a0ae9700fbf795c943 P e8b65b15973c4d7d896a96567f97caee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b66124a6ca6244748b38ebb07c6ca40e" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41515 } } peers { permanent_uuid: "e8b65b15973c4d7d896a96567f97caee" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 38009 } } peers { permanent_uuid: "7136de9ce16f4703a928d1429952cdea" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 45057 } }
I20260811 09:07:51.081967 1269 ts_tablet_manager.cc:1428] T 5497ddad181d48a0ae9700fbf795c943 P e8b65b15973c4d7d896a96567f97caee: Time spent starting tablet: real 0.004s user 0.001s sys 0.002s
I20260811 09:07:51.083251 1271 tablet_bootstrap.cc:492] T 5497ddad181d48a0ae9700fbf795c943 P b66124a6ca6244748b38ebb07c6ca40e: No bootstrap required, opened a new log
I20260811 09:07:51.083423 1271 ts_tablet_manager.cc:1397] T 5497ddad181d48a0ae9700fbf795c943 P b66124a6ca6244748b38ebb07c6ca40e: Time spent bootstrapping tablet: real 0.023s user 0.011s sys 0.008s
I20260811 09:07:51.084276 1271 raft_consensus.cc:348] T 5497ddad181d48a0ae9700fbf795c943 P b66124a6ca6244748b38ebb07c6ca40e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b66124a6ca6244748b38ebb07c6ca40e" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41515 } } peers { permanent_uuid: "e8b65b15973c4d7d896a96567f97caee" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 38009 } } peers { permanent_uuid: "7136de9ce16f4703a928d1429952cdea" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 45057 } }
I20260811 09:07:51.084522 1275 tablet_bootstrap.cc:492] T 5497ddad181d48a0ae9700fbf795c943 P 7136de9ce16f4703a928d1429952cdea: No bootstrap required, opened a new log
I20260811 09:07:51.084668 1275 ts_tablet_manager.cc:1397] T 5497ddad181d48a0ae9700fbf795c943 P 7136de9ce16f4703a928d1429952cdea: Time spent bootstrapping tablet: real 0.013s user 0.010s sys 0.000s
I20260811 09:07:51.084678 1271 raft_consensus.cc:374] T 5497ddad181d48a0ae9700fbf795c943 P b66124a6ca6244748b38ebb07c6ca40e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:07:51.084805 1271 raft_consensus.cc:729] T 5497ddad181d48a0ae9700fbf795c943 P b66124a6ca6244748b38ebb07c6ca40e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b66124a6ca6244748b38ebb07c6ca40e, State: Initialized, Role: FOLLOWER
I20260811 09:07:51.085142 1271 consensus_queue.cc:260] T 5497ddad181d48a0ae9700fbf795c943 P b66124a6ca6244748b38ebb07c6ca40e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b66124a6ca6244748b38ebb07c6ca40e" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41515 } } peers { permanent_uuid: "e8b65b15973c4d7d896a96567f97caee" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 38009 } } peers { permanent_uuid: "7136de9ce16f4703a928d1429952cdea" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 45057 } }
I20260811 09:07:51.085539 1275 raft_consensus.cc:348] T 5497ddad181d48a0ae9700fbf795c943 P 7136de9ce16f4703a928d1429952cdea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b66124a6ca6244748b38ebb07c6ca40e" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41515 } } peers { permanent_uuid: "e8b65b15973c4d7d896a96567f97caee" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 38009 } } peers { permanent_uuid: "7136de9ce16f4703a928d1429952cdea" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 45057 } }
I20260811 09:07:51.085840 1275 raft_consensus.cc:374] T 5497ddad181d48a0ae9700fbf795c943 P 7136de9ce16f4703a928d1429952cdea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:07:51.085964 1275 raft_consensus.cc:729] T 5497ddad181d48a0ae9700fbf795c943 P 7136de9ce16f4703a928d1429952cdea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7136de9ce16f4703a928d1429952cdea, State: Initialized, Role: FOLLOWER
I20260811 09:07:51.086293 1271 ts_tablet_manager.cc:1428] T 5497ddad181d48a0ae9700fbf795c943 P b66124a6ca6244748b38ebb07c6ca40e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260811 09:07:51.086248 1275 consensus_queue.cc:260] T 5497ddad181d48a0ae9700fbf795c943 P 7136de9ce16f4703a928d1429952cdea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b66124a6ca6244748b38ebb07c6ca40e" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41515 } } peers { permanent_uuid: "e8b65b15973c4d7d896a96567f97caee" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 38009 } } peers { permanent_uuid: "7136de9ce16f4703a928d1429952cdea" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 45057 } }
I20260811 09:07:51.086884 1275 ts_tablet_manager.cc:1428] T 5497ddad181d48a0ae9700fbf795c943 P 7136de9ce16f4703a928d1429952cdea: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 09:07:51.337107 1277 raft_consensus.cc:482] T 5497ddad181d48a0ae9700fbf795c943 P b66124a6ca6244748b38ebb07c6ca40e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:07:51.337357 1277 raft_consensus.cc:504] T 5497ddad181d48a0ae9700fbf795c943 P b66124a6ca6244748b38ebb07c6ca40e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b66124a6ca6244748b38ebb07c6ca40e" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41515 } } peers { permanent_uuid: "e8b65b15973c4d7d896a96567f97caee" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 38009 } } peers { permanent_uuid: "7136de9ce16f4703a928d1429952cdea" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 45057 } }
I20260811 09:07:51.339076 1277 leader_election.cc:288] T 5497ddad181d48a0ae9700fbf795c943 P b66124a6ca6244748b38ebb07c6ca40e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8b65b15973c4d7d896a96567f97caee (127.28.154.130:38009), 7136de9ce16f4703a928d1429952cdea (127.28.154.131:45057)
I20260811 09:07:51.348028 1284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:51.339534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.129:38709 (local address 127.28.154.130:38009)
0811 09:07:51.340768 (+ 1234us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:51.340774 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:51.340800 (+ 26us) server_negotiation.cc:407] Connection header received
0811 09:07:51.340878 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:51.340884 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:51.340992 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:51.341091 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:07:51.341705 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:51.342634 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:51.345817 (+ 3183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:51.346850 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:51.347059 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:51.347198 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:51.347502 (+ 304us) server_negotiation.cc:299] Negotiation successful
0811 09:07:51.347654 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1078,"thread_start_us":1087,"threads_started":1}
I20260811 09:07:51.348013 1282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:51.339360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.130:38009 (local address 127.28.154.129:38709)
0811 09:07:51.340043 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:51.340066 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 09:07:51.340318 (+ 252us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:07:51.341120 (+ 802us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:07:51.341126 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:07:51.341156 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:07:51.341482 (+ 326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:51.341497 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:51.342934 (+ 1437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:51.342944 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:51.345525 (+ 2581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:51.345550 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:51.347097 (+ 1547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:51.347105 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:51.347296 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:51.347334 (+ 38us) client_negotiation.cc:769] Sending connection context
0811 09:07:51.347436 (+ 102us) client_negotiation.cc:240] Negotiation successful
0811 09:07:51.347606 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":537,"mutex_wait_us":1,"thread_start_us":373,"threads_started":1}
I20260811 09:07:51.348584 1285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:51.340247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.131:45057 (local address 127.28.154.129:46359)
0811 09:07:51.340864 (+ 617us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:51.340884 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 09:07:51.340989 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:07:51.341492 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:07:51.341495 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:07:51.341516 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:07:51.341947 (+ 431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:51.341959 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:51.343758 (+ 1799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:51.343766 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:51.346022 (+ 2256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:51.346040 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:51.347542 (+ 1502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:51.347549 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:51.347789 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:51.347824 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 09:07:51.347934 (+ 110us) client_negotiation.cc:240] Negotiation successful
0811 09:07:51.348079 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":503,"mutex_wait_us":45,"thread_start_us":359,"threads_started":1}
I20260811 09:07:51.348618 1286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:51.340469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.129:46359 (local address 127.28.154.131:45057)
0811 09:07:51.341176 (+ 707us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:51.341181 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:51.341202 (+ 21us) server_negotiation.cc:407] Connection header received
0811 09:07:51.341272 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:51.341277 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:51.341344 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:51.341456 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:07:51.342140 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:51.343507 (+ 1367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:51.346228 (+ 2721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:51.347290 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:51.347488 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:51.347558 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:51.347977 (+ 419us) server_negotiation.cc:299] Negotiation successful
0811 09:07:51.348128 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":583,"thread_start_us":449,"threads_started":1}
I20260811 09:07:51.349047 685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5497ddad181d48a0ae9700fbf795c943" candidate_uuid: "b66124a6ca6244748b38ebb07c6ca40e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8b65b15973c4d7d896a96567f97caee" is_pre_election: true
I20260811 09:07:51.349217 952 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5497ddad181d48a0ae9700fbf795c943" candidate_uuid: "b66124a6ca6244748b38ebb07c6ca40e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7136de9ce16f4703a928d1429952cdea" is_pre_election: true
I20260811 09:07:51.349339 685 raft_consensus.cc:2454] T 5497ddad181d48a0ae9700fbf795c943 P e8b65b15973c4d7d896a96567f97caee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b66124a6ca6244748b38ebb07c6ca40e in term 0.
I20260811 09:07:51.349499 952 raft_consensus.cc:2454] T 5497ddad181d48a0ae9700fbf795c943 P 7136de9ce16f4703a928d1429952cdea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b66124a6ca6244748b38ebb07c6ca40e in term 0.
I20260811 09:07:51.350059 32738 leader_election.cc:302] T 5497ddad181d48a0ae9700fbf795c943 P b66124a6ca6244748b38ebb07c6ca40e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b66124a6ca6244748b38ebb07c6ca40e, e8b65b15973c4d7d896a96567f97caee; no voters:
I20260811 09:07:51.350543 1277 raft_consensus.cc:2781] T 5497ddad181d48a0ae9700fbf795c943 P b66124a6ca6244748b38ebb07c6ca40e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 09:07:51.350674 1277 raft_consensus.cc:482] T 5497ddad181d48a0ae9700fbf795c943 P b66124a6ca6244748b38ebb07c6ca40e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 09:07:51.350749 1277 raft_consensus.cc:3037] T 5497ddad181d48a0ae9700fbf795c943 P b66124a6ca6244748b38ebb07c6ca40e [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:07:51.352949 1277 raft_consensus.cc:504] T 5497ddad181d48a0ae9700fbf795c943 P b66124a6ca6244748b38ebb07c6ca40e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b66124a6ca6244748b38ebb07c6ca40e" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41515 } } peers { permanent_uuid: "e8b65b15973c4d7d896a96567f97caee" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 38009 } } peers { permanent_uuid: "7136de9ce16f4703a928d1429952cdea" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 45057 } }
I20260811 09:07:51.353565 1277 leader_election.cc:288] T 5497ddad181d48a0ae9700fbf795c943 P b66124a6ca6244748b38ebb07c6ca40e [CANDIDATE]: Term 1 election: Requested vote from peers e8b65b15973c4d7d896a96567f97caee (127.28.154.130:38009), 7136de9ce16f4703a928d1429952cdea (127.28.154.131:45057)
I20260811 09:07:51.354164 685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5497ddad181d48a0ae9700fbf795c943" candidate_uuid: "b66124a6ca6244748b38ebb07c6ca40e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8b65b15973c4d7d896a96567f97caee"
I20260811 09:07:51.354224 952 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5497ddad181d48a0ae9700fbf795c943" candidate_uuid: "b66124a6ca6244748b38ebb07c6ca40e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7136de9ce16f4703a928d1429952cdea"
I20260811 09:07:51.354414 952 raft_consensus.cc:3037] T 5497ddad181d48a0ae9700fbf795c943 P 7136de9ce16f4703a928d1429952cdea [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:07:51.354472 685 raft_consensus.cc:3037] T 5497ddad181d48a0ae9700fbf795c943 P e8b65b15973c4d7d896a96567f97caee [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:07:51.357431 952 raft_consensus.cc:2454] T 5497ddad181d48a0ae9700fbf795c943 P 7136de9ce16f4703a928d1429952cdea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b66124a6ca6244748b38ebb07c6ca40e in term 1.
I20260811 09:07:51.357429 685 raft_consensus.cc:2454] T 5497ddad181d48a0ae9700fbf795c943 P e8b65b15973c4d7d896a96567f97caee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b66124a6ca6244748b38ebb07c6ca40e in term 1.
I20260811 09:07:51.357913 32738 leader_election.cc:302] T 5497ddad181d48a0ae9700fbf795c943 P b66124a6ca6244748b38ebb07c6ca40e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7136de9ce16f4703a928d1429952cdea, b66124a6ca6244748b38ebb07c6ca40e; no voters:
I20260811 09:07:51.358230 1277 raft_consensus.cc:2781] T 5497ddad181d48a0ae9700fbf795c943 P b66124a6ca6244748b38ebb07c6ca40e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:07:51.358772 1277 raft_consensus.cc:686] T 5497ddad181d48a0ae9700fbf795c943 P b66124a6ca6244748b38ebb07c6ca40e [term 1 LEADER]: Becoming Leader. State: Replica: b66124a6ca6244748b38ebb07c6ca40e, State: Running, Role: LEADER
I20260811 09:07:51.359198 1277 consensus_queue.cc:237] T 5497ddad181d48a0ae9700fbf795c943 P b66124a6ca6244748b38ebb07c6ca40e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b66124a6ca6244748b38ebb07c6ca40e" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41515 } } peers { permanent_uuid: "e8b65b15973c4d7d896a96567f97caee" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 38009 } } peers { permanent_uuid: "7136de9ce16f4703a928d1429952cdea" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 45057 } }
I20260811 09:07:51.362599 32566 catalog_manager.cc:5495] T 5497ddad181d48a0ae9700fbf795c943 P b66124a6ca6244748b38ebb07c6ca40e reported cstate change: term changed from 0 to 1, leader changed from <none> to b66124a6ca6244748b38ebb07c6ca40e (127.28.154.129). New cstate: current_term: 1 leader_uuid: "b66124a6ca6244748b38ebb07c6ca40e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b66124a6ca6244748b38ebb07c6ca40e" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e8b65b15973c4d7d896a96567f97caee" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 38009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7136de9ce16f4703a928d1429952cdea" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 45057 } health_report { overall_health: UNKNOWN } } }
I20260811 09:07:51.446071 1295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:51.439774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49256 (local address 127.28.154.129:41515)
0811 09:07:51.440386 (+ 612us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:51.440390 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:51.440409 (+ 19us) server_negotiation.cc:407] Connection header received
0811 09:07:51.440499 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:51.440502 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:51.440577 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:51.440729 (+ 152us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:07:51.441123 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:51.441947 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:51.443329 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:51.443782 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:51.443881 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:51.444510 (+ 629us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:07:51.445319 (+ 809us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:07:51.445440 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:51.445613 (+ 173us) server_negotiation.cc:299] Negotiation successful
0811 09:07:51.445695 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":481,"thread_start_us":338,"threads_started":1}
I20260811 09:07:51.462009 1286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:51.456491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.129:55417 (local address 127.28.154.131:45057)
0811 09:07:51.456685 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:51.456690 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:51.457175 (+ 485us) server_negotiation.cc:407] Connection header received
0811 09:07:51.457345 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:51.457348 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:51.457416 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:51.457498 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:07:51.458063 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:51.458746 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:51.460222 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:51.460701 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:51.460823 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:51.461278 (+ 455us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:07:51.461434 (+ 156us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:07:51.461516 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:51.461668 (+ 152us) server_negotiation.cc:299] Negotiation successful
0811 09:07:51.461748 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 09:07:51.462162 1303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:51.456292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.131:45057 (local address 127.28.154.129:55417)
0811 09:07:51.457090 (+ 798us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:51.457107 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 09:07:51.457233 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:07:51.457522 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:07:51.457526 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:07:51.457537 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 09:07:51.457906 (+ 369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:51.457917 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:51.458874 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:51.458879 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:51.460078 (+ 1199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:51.460092 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:51.460891 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:51.460897 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:51.461038 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:51.461103 (+ 65us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 09:07:51.461532 (+ 429us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 09:07:51.461562 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 09:07:51.461635 (+ 73us) client_negotiation.cc:240] Negotiation successful
0811 09:07:51.461770 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":622,"thread_start_us":453,"threads_started":1}
I20260811 09:07:51.472098 685 raft_consensus.cc:1261] T 5497ddad181d48a0ae9700fbf795c943 P e8b65b15973c4d7d896a96567f97caee [term 1 FOLLOWER]: Refusing update from remote peer b66124a6ca6244748b38ebb07c6ca40e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:07:51.472098 952 raft_consensus.cc:1261] T 5497ddad181d48a0ae9700fbf795c943 P 7136de9ce16f4703a928d1429952cdea [term 1 FOLLOWER]: Refusing update from remote peer b66124a6ca6244748b38ebb07c6ca40e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:07:51.472818 1289 consensus_queue.cc:1035] T 5497ddad181d48a0ae9700fbf795c943 P b66124a6ca6244748b38ebb07c6ca40e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8b65b15973c4d7d896a96567f97caee" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 38009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:07:51.473201 1277 consensus_queue.cc:1035] T 5497ddad181d48a0ae9700fbf795c943 P b66124a6ca6244748b38ebb07c6ca40e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7136de9ce16f4703a928d1429952cdea" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 45057 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:07:51.499564 1295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:51.494223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.131:46053 (local address 127.28.154.129:41515)
0811 09:07:51.494462 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:51.494468 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:51.494719 (+ 251us) server_negotiation.cc:407] Connection header received
0811 09:07:51.494910 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:51.494916 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:51.494988 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:51.495066 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:07:51.495501 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:51.496199 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:51.497622 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:51.498248 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:51.498378 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:51.498861 (+ 483us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:07:51.498989 (+ 128us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:07:51.499077 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:51.499270 (+ 193us) server_negotiation.cc:299] Negotiation successful
0811 09:07:51.499340 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 09:07:51.499632 1312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:51.494051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.129:41515 (local address 127.28.154.131:46053)
0811 09:07:51.494638 (+ 587us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:51.494653 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 09:07:51.494777 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:07:51.495097 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:07:51.495102 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:07:51.495114 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 09:07:51.495362 (+ 248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:51.495372 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:51.496356 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:51.496362 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:51.497501 (+ 1139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:51.497514 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:51.498456 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:51.498463 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:51.498622 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:51.498661 (+ 39us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 09:07:51.499130 (+ 469us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 09:07:51.499143 (+ 13us) client_negotiation.cc:769] Sending connection context
0811 09:07:51.499218 (+ 75us) client_negotiation.cc:240] Negotiation successful
0811 09:07:51.499329 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":256,"threads_started":1}
I20260811 09:07:51.534134 32566 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47770:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:07:51.534499 32566 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47770:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:07:51.537963 32566 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 49c59055f30d4f5e8fc4a7bfebbdd512: Sending DeleteTablet for 3 replicas of tablet 5497ddad181d48a0ae9700fbf795c943
I20260811 09:07:51.539139 910 tablet_service.cc:1514] Processing DeleteTablet for tablet 5497ddad181d48a0ae9700fbf795c943 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:07:51 UTC) from {username='kudu', principal='kudu/127.28.154.190@KRBTEST.COM'} at 127.0.0.1:37208
I20260811 09:07:51.539359 641 tablet_service.cc:1514] Processing DeleteTablet for tablet 5497ddad181d48a0ae9700fbf795c943 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:07:51 UTC) from {username='kudu', principal='kudu/127.28.154.190@KRBTEST.COM'} at 127.0.0.1:59830
I20260811 09:07:51.539371 361 tablet_service.cc:1514] Processing DeleteTablet for tablet 5497ddad181d48a0ae9700fbf795c943 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:07:51 UTC) from {username='kudu', principal='kudu/127.28.154.190@KRBTEST.COM'} at 127.0.0.1:49252
I20260811 09:07:51.545252 32566 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47770:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 09:07:51.546113 32566 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 09:07:51.550598 1327 tablet_replica.cc:330] T 5497ddad181d48a0ae9700fbf795c943 P b66124a6ca6244748b38ebb07c6ca40e: stopping tablet replica
I20260811 09:07:51.550832 1327 raft_consensus.cc:2229] T 5497ddad181d48a0ae9700fbf795c943 P b66124a6ca6244748b38ebb07c6ca40e [term 1 LEADER]: Raft consensus shutting down.
I20260811 09:07:51.551545 1327 raft_consensus.cc:2258] T 5497ddad181d48a0ae9700fbf795c943 P b66124a6ca6244748b38ebb07c6ca40e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:07:51.554168 1327 ts_tablet_manager.cc:1905] T 5497ddad181d48a0ae9700fbf795c943 P b66124a6ca6244748b38ebb07c6ca40e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:07:51.557525 908 tablet_service.cc:1467] Processing CreateTablet for tablet ffb68a8b60e74c15b2b45b7cbca7e0e8 (DEFAULT_TABLE table=test-table [id=75dffa0a5561436ebb8a0956d33fe177]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:07:51.558074 908 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ffb68a8b60e74c15b2b45b7cbca7e0e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:07:51.558297 361 tablet_service.cc:1467] Processing CreateTablet for tablet ffb68a8b60e74c15b2b45b7cbca7e0e8 (DEFAULT_TABLE table=test-table [id=75dffa0a5561436ebb8a0956d33fe177]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:07:51.558751 361 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ffb68a8b60e74c15b2b45b7cbca7e0e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:07:51.559005 644 tablet_service.cc:1467] Processing CreateTablet for tablet ffb68a8b60e74c15b2b45b7cbca7e0e8 (DEFAULT_TABLE table=test-table [id=75dffa0a5561436ebb8a0956d33fe177]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:07:51.559484 644 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ffb68a8b60e74c15b2b45b7cbca7e0e8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:07:51.564604 1275 tablet_bootstrap.cc:492] T ffb68a8b60e74c15b2b45b7cbca7e0e8 P 7136de9ce16f4703a928d1429952cdea: Bootstrap starting.
I20260811 09:07:51.566129 1269 tablet_bootstrap.cc:492] T ffb68a8b60e74c15b2b45b7cbca7e0e8 P e8b65b15973c4d7d896a96567f97caee: Bootstrap starting.
I20260811 09:07:51.566253 1275 tablet_bootstrap.cc:654] T ffb68a8b60e74c15b2b45b7cbca7e0e8 P 7136de9ce16f4703a928d1429952cdea: Neither blocks nor log segments found. Creating new log.
I20260811 09:07:51.567461 1271 tablet_bootstrap.cc:492] T ffb68a8b60e74c15b2b45b7cbca7e0e8 P b66124a6ca6244748b38ebb07c6ca40e: Bootstrap starting.
I20260811 09:07:51.567860 1269 tablet_bootstrap.cc:654] T ffb68a8b60e74c15b2b45b7cbca7e0e8 P e8b65b15973c4d7d896a96567f97caee: Neither blocks nor log segments found. Creating new log.
I20260811 09:07:51.569260 1271 tablet_bootstrap.cc:654] T ffb68a8b60e74c15b2b45b7cbca7e0e8 P b66124a6ca6244748b38ebb07c6ca40e: Neither blocks nor log segments found. Creating new log.
I20260811 09:07:51.569409 1324 tablet_replica.cc:330] T 5497ddad181d48a0ae9700fbf795c943 P 7136de9ce16f4703a928d1429952cdea: stopping tablet replica
I20260811 09:07:51.569594 1324 raft_consensus.cc:2229] T 5497ddad181d48a0ae9700fbf795c943 P 7136de9ce16f4703a928d1429952cdea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:07:51.569875 1324 raft_consensus.cc:2258] T 5497ddad181d48a0ae9700fbf795c943 P 7136de9ce16f4703a928d1429952cdea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:07:51.572281 1324 ts_tablet_manager.cc:1905] T 5497ddad181d48a0ae9700fbf795c943 P 7136de9ce16f4703a928d1429952cdea: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:07:51.577301 1269 tablet_bootstrap.cc:492] T ffb68a8b60e74c15b2b45b7cbca7e0e8 P e8b65b15973c4d7d896a96567f97caee: No bootstrap required, opened a new log
I20260811 09:07:51.577580 1269 ts_tablet_manager.cc:1397] T ffb68a8b60e74c15b2b45b7cbca7e0e8 P e8b65b15973c4d7d896a96567f97caee: Time spent bootstrapping tablet: real 0.012s user 0.009s sys 0.001s
I20260811 09:07:51.578460 1269 raft_consensus.cc:348] T ffb68a8b60e74c15b2b45b7cbca7e0e8 P e8b65b15973c4d7d896a96567f97caee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b66124a6ca6244748b38ebb07c6ca40e" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41515 } } peers { permanent_uuid: "e8b65b15973c4d7d896a96567f97caee" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 38009 } } peers { permanent_uuid: "7136de9ce16f4703a928d1429952cdea" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 45057 } }
I20260811 09:07:51.578890 1269 raft_consensus.cc:374] T ffb68a8b60e74c15b2b45b7cbca7e0e8 P e8b65b15973c4d7d896a96567f97caee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:07:51.579025 1269 raft_consensus.cc:729] T ffb68a8b60e74c15b2b45b7cbca7e0e8 P e8b65b15973c4d7d896a96567f97caee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8b65b15973c4d7d896a96567f97caee, State: Initialized, Role: FOLLOWER
I20260811 09:07:51.579401 1269 consensus_queue.cc:260] T ffb68a8b60e74c15b2b45b7cbca7e0e8 P e8b65b15973c4d7d896a96567f97caee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b66124a6ca6244748b38ebb07c6ca40e" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41515 } } peers { permanent_uuid: "e8b65b15973c4d7d896a96567f97caee" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 38009 } } peers { permanent_uuid: "7136de9ce16f4703a928d1429952cdea" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 45057 } }
I20260811 09:07:51.581014 1269 ts_tablet_manager.cc:1428] T ffb68a8b60e74c15b2b45b7cbca7e0e8 P e8b65b15973c4d7d896a96567f97caee: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260811 09:07:51.586849 1324 ts_tablet_manager.cc:1918] T 5497ddad181d48a0ae9700fbf795c943 P 7136de9ce16f4703a928d1429952cdea: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 09:07:51.587090 1324 log.cc:1196] T 5497ddad181d48a0ae9700fbf795c943 P 7136de9ce16f4703a928d1429952cdea: Deleting WAL directory at /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439250192320-29290-0/minicluster-data/ts-2/wal/wals/5497ddad181d48a0ae9700fbf795c943
I20260811 09:07:51.587654 1324 ts_tablet_manager.cc:1939] T 5497ddad181d48a0ae9700fbf795c943 P 7136de9ce16f4703a928d1429952cdea: Deleting consensus metadata
I20260811 09:07:51.589232 32534 catalog_manager.cc:4841] TS 7136de9ce16f4703a928d1429952cdea (127.28.154.131:45057): tablet 5497ddad181d48a0ae9700fbf795c943 (table test-table [id=9800071265fa4bee937aaf52c659028c]) successfully deleted
I20260811 09:07:51.592028 1271 tablet_bootstrap.cc:492] T ffb68a8b60e74c15b2b45b7cbca7e0e8 P b66124a6ca6244748b38ebb07c6ca40e: No bootstrap required, opened a new log
I20260811 09:07:51.592217 1271 ts_tablet_manager.cc:1397] T ffb68a8b60e74c15b2b45b7cbca7e0e8 P b66124a6ca6244748b38ebb07c6ca40e: Time spent bootstrapping tablet: real 0.025s user 0.006s sys 0.005s
I20260811 09:07:51.593118 1271 raft_consensus.cc:348] T ffb68a8b60e74c15b2b45b7cbca7e0e8 P b66124a6ca6244748b38ebb07c6ca40e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b66124a6ca6244748b38ebb07c6ca40e" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41515 } } peers { permanent_uuid: "e8b65b15973c4d7d896a96567f97caee" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 38009 } } peers { permanent_uuid: "7136de9ce16f4703a928d1429952cdea" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 45057 } }
I20260811 09:07:51.593497 1271 raft_consensus.cc:374] T ffb68a8b60e74c15b2b45b7cbca7e0e8 P b66124a6ca6244748b38ebb07c6ca40e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:07:51.593653 1271 raft_consensus.cc:729] T ffb68a8b60e74c15b2b45b7cbca7e0e8 P b66124a6ca6244748b38ebb07c6ca40e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b66124a6ca6244748b38ebb07c6ca40e, State: Initialized, Role: FOLLOWER
I20260811 09:07:51.594041 1271 consensus_queue.cc:260] T ffb68a8b60e74c15b2b45b7cbca7e0e8 P b66124a6ca6244748b38ebb07c6ca40e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b66124a6ca6244748b38ebb07c6ca40e" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41515 } } peers { permanent_uuid: "e8b65b15973c4d7d896a96567f97caee" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 38009 } } peers { permanent_uuid: "7136de9ce16f4703a928d1429952cdea" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 45057 } }
I20260811 09:07:51.594779 1275 tablet_bootstrap.cc:492] T ffb68a8b60e74c15b2b45b7cbca7e0e8 P 7136de9ce16f4703a928d1429952cdea: No bootstrap required, opened a new log
I20260811 09:07:51.594961 1275 ts_tablet_manager.cc:1397] T ffb68a8b60e74c15b2b45b7cbca7e0e8 P 7136de9ce16f4703a928d1429952cdea: Time spent bootstrapping tablet: real 0.030s user 0.004s sys 0.008s
I20260811 09:07:51.597486 1271 ts_tablet_manager.cc:1428] T ffb68a8b60e74c15b2b45b7cbca7e0e8 P b66124a6ca6244748b38ebb07c6ca40e: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260811 09:07:51.600673 1275 raft_consensus.cc:348] T ffb68a8b60e74c15b2b45b7cbca7e0e8 P 7136de9ce16f4703a928d1429952cdea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b66124a6ca6244748b38ebb07c6ca40e" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41515 } } peers { permanent_uuid: "e8b65b15973c4d7d896a96567f97caee" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 38009 } } peers { permanent_uuid: "7136de9ce16f4703a928d1429952cdea" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 45057 } }
I20260811 09:07:51.600898 1275 raft_consensus.cc:374] T ffb68a8b60e74c15b2b45b7cbca7e0e8 P 7136de9ce16f4703a928d1429952cdea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:07:51.600988 1275 raft_consensus.cc:729] T ffb68a8b60e74c15b2b45b7cbca7e0e8 P 7136de9ce16f4703a928d1429952cdea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7136de9ce16f4703a928d1429952cdea, State: Initialized, Role: FOLLOWER
I20260811 09:07:51.601239 1275 consensus_queue.cc:260] T ffb68a8b60e74c15b2b45b7cbca7e0e8 P 7136de9ce16f4703a928d1429952cdea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b66124a6ca6244748b38ebb07c6ca40e" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41515 } } peers { permanent_uuid: "e8b65b15973c4d7d896a96567f97caee" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 38009 } } peers { permanent_uuid: "7136de9ce16f4703a928d1429952cdea" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 45057 } }
I20260811 09:07:51.601801 1275 ts_tablet_manager.cc:1428] T ffb68a8b60e74c15b2b45b7cbca7e0e8 P 7136de9ce16f4703a928d1429952cdea: Time spent starting tablet: real 0.007s user 0.001s sys 0.000s
I20260811 09:07:51.602663 1326 tablet_replica.cc:330] T 5497ddad181d48a0ae9700fbf795c943 P e8b65b15973c4d7d896a96567f97caee: stopping tablet replica
I20260811 09:07:51.603849 1326 raft_consensus.cc:2229] T 5497ddad181d48a0ae9700fbf795c943 P e8b65b15973c4d7d896a96567f97caee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:07:51.604120 1326 raft_consensus.cc:2258] T 5497ddad181d48a0ae9700fbf795c943 P e8b65b15973c4d7d896a96567f97caee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:07:51.605351 1326 ts_tablet_manager.cc:1905] T 5497ddad181d48a0ae9700fbf795c943 P e8b65b15973c4d7d896a96567f97caee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:07:51.610481 1327 ts_tablet_manager.cc:1918] T 5497ddad181d48a0ae9700fbf795c943 P b66124a6ca6244748b38ebb07c6ca40e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 09:07:51.610654 1327 log.cc:1196] T 5497ddad181d48a0ae9700fbf795c943 P b66124a6ca6244748b38ebb07c6ca40e: Deleting WAL directory at /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439250192320-29290-0/minicluster-data/ts-0/wal/wals/5497ddad181d48a0ae9700fbf795c943
I20260811 09:07:51.610728 1326 ts_tablet_manager.cc:1918] T 5497ddad181d48a0ae9700fbf795c943 P e8b65b15973c4d7d896a96567f97caee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 09:07:51.610886 1326 log.cc:1196] T 5497ddad181d48a0ae9700fbf795c943 P e8b65b15973c4d7d896a96567f97caee: Deleting WAL directory at /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439250192320-29290-0/minicluster-data/ts-1/wal/wals/5497ddad181d48a0ae9700fbf795c943
I20260811 09:07:51.611187 1327 ts_tablet_manager.cc:1939] T 5497ddad181d48a0ae9700fbf795c943 P b66124a6ca6244748b38ebb07c6ca40e: Deleting consensus metadata
I20260811 09:07:51.611306 1326 ts_tablet_manager.cc:1939] T 5497ddad181d48a0ae9700fbf795c943 P e8b65b15973c4d7d896a96567f97caee: Deleting consensus metadata
I20260811 09:07:51.612370 32534 catalog_manager.cc:4841] TS b66124a6ca6244748b38ebb07c6ca40e (127.28.154.129:41515): tablet 5497ddad181d48a0ae9700fbf795c943 (table test-table [id=9800071265fa4bee937aaf52c659028c]) successfully deleted
I20260811 09:07:51.612694 32534 catalog_manager.cc:4841] TS e8b65b15973c4d7d896a96567f97caee (127.28.154.130:38009): tablet 5497ddad181d48a0ae9700fbf795c943 (table test-table [id=9800071265fa4bee937aaf52c659028c]) successfully deleted
I20260811 09:07:51.663904 1316 raft_consensus.cc:482] T ffb68a8b60e74c15b2b45b7cbca7e0e8 P 7136de9ce16f4703a928d1429952cdea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:07:51.664181 1316 raft_consensus.cc:504] T ffb68a8b60e74c15b2b45b7cbca7e0e8 P 7136de9ce16f4703a928d1429952cdea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b66124a6ca6244748b38ebb07c6ca40e" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41515 } } peers { permanent_uuid: "e8b65b15973c4d7d896a96567f97caee" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 38009 } } peers { permanent_uuid: "7136de9ce16f4703a928d1429952cdea" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 45057 } }
I20260811 09:07:51.665048 1316 leader_election.cc:288] T ffb68a8b60e74c15b2b45b7cbca7e0e8 P 7136de9ce16f4703a928d1429952cdea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b66124a6ca6244748b38ebb07c6ca40e (127.28.154.129:41515), e8b65b15973c4d7d896a96567f97caee (127.28.154.130:38009)
I20260811 09:07:51.665726 685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffb68a8b60e74c15b2b45b7cbca7e0e8" candidate_uuid: "7136de9ce16f4703a928d1429952cdea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8b65b15973c4d7d896a96567f97caee" is_pre_election: true
I20260811 09:07:51.665668 407 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffb68a8b60e74c15b2b45b7cbca7e0e8" candidate_uuid: "7136de9ce16f4703a928d1429952cdea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b66124a6ca6244748b38ebb07c6ca40e" is_pre_election: true
I20260811 09:07:51.666010 407 raft_consensus.cc:2454] T ffb68a8b60e74c15b2b45b7cbca7e0e8 P b66124a6ca6244748b38ebb07c6ca40e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7136de9ce16f4703a928d1429952cdea in term 0.
I20260811 09:07:51.666018 685 raft_consensus.cc:2454] T ffb68a8b60e74c15b2b45b7cbca7e0e8 P e8b65b15973c4d7d896a96567f97caee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7136de9ce16f4703a928d1429952cdea in term 0.
I20260811 09:07:51.666554 802 leader_election.cc:302] T ffb68a8b60e74c15b2b45b7cbca7e0e8 P 7136de9ce16f4703a928d1429952cdea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7136de9ce16f4703a928d1429952cdea, b66124a6ca6244748b38ebb07c6ca40e; no voters:
I20260811 09:07:51.666889 1316 raft_consensus.cc:2781] T ffb68a8b60e74c15b2b45b7cbca7e0e8 P 7136de9ce16f4703a928d1429952cdea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 09:07:51.667007 1316 raft_consensus.cc:482] T ffb68a8b60e74c15b2b45b7cbca7e0e8 P 7136de9ce16f4703a928d1429952cdea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 09:07:51.667094 1316 raft_consensus.cc:3037] T ffb68a8b60e74c15b2b45b7cbca7e0e8 P 7136de9ce16f4703a928d1429952cdea [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:07:51.669332 1316 raft_consensus.cc:504] T ffb68a8b60e74c15b2b45b7cbca7e0e8 P 7136de9ce16f4703a928d1429952cdea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b66124a6ca6244748b38ebb07c6ca40e" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41515 } } peers { permanent_uuid: "e8b65b15973c4d7d896a96567f97caee" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 38009 } } peers { permanent_uuid: "7136de9ce16f4703a928d1429952cdea" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 45057 } }
I20260811 09:07:51.669932 1316 leader_election.cc:288] T ffb68a8b60e74c15b2b45b7cbca7e0e8 P 7136de9ce16f4703a928d1429952cdea [CANDIDATE]: Term 1 election: Requested vote from peers b66124a6ca6244748b38ebb07c6ca40e (127.28.154.129:41515), e8b65b15973c4d7d896a96567f97caee (127.28.154.130:38009)
I20260811 09:07:51.670559 685 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffb68a8b60e74c15b2b45b7cbca7e0e8" candidate_uuid: "7136de9ce16f4703a928d1429952cdea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8b65b15973c4d7d896a96567f97caee"
I20260811 09:07:51.670558 407 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ffb68a8b60e74c15b2b45b7cbca7e0e8" candidate_uuid: "7136de9ce16f4703a928d1429952cdea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b66124a6ca6244748b38ebb07c6ca40e"
I20260811 09:07:51.670790 685 raft_consensus.cc:3037] T ffb68a8b60e74c15b2b45b7cbca7e0e8 P e8b65b15973c4d7d896a96567f97caee [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:07:51.670843 407 raft_consensus.cc:3037] T ffb68a8b60e74c15b2b45b7cbca7e0e8 P b66124a6ca6244748b38ebb07c6ca40e [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:07:51.673460 685 raft_consensus.cc:2454] T ffb68a8b60e74c15b2b45b7cbca7e0e8 P e8b65b15973c4d7d896a96567f97caee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7136de9ce16f4703a928d1429952cdea in term 1.
I20260811 09:07:51.673897 802 leader_election.cc:302] T ffb68a8b60e74c15b2b45b7cbca7e0e8 P 7136de9ce16f4703a928d1429952cdea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7136de9ce16f4703a928d1429952cdea, e8b65b15973c4d7d896a96567f97caee; no voters:
I20260811 09:07:51.674115 407 raft_consensus.cc:2454] T ffb68a8b60e74c15b2b45b7cbca7e0e8 P b66124a6ca6244748b38ebb07c6ca40e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7136de9ce16f4703a928d1429952cdea in term 1.
I20260811 09:07:51.674182 1316 raft_consensus.cc:2781] T ffb68a8b60e74c15b2b45b7cbca7e0e8 P 7136de9ce16f4703a928d1429952cdea [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:07:51.674325 1316 raft_consensus.cc:686] T ffb68a8b60e74c15b2b45b7cbca7e0e8 P 7136de9ce16f4703a928d1429952cdea [term 1 LEADER]: Becoming Leader. State: Replica: 7136de9ce16f4703a928d1429952cdea, State: Running, Role: LEADER
I20260811 09:07:51.674638 1316 consensus_queue.cc:237] T ffb68a8b60e74c15b2b45b7cbca7e0e8 P 7136de9ce16f4703a928d1429952cdea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b66124a6ca6244748b38ebb07c6ca40e" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41515 } } peers { permanent_uuid: "e8b65b15973c4d7d896a96567f97caee" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 38009 } } peers { permanent_uuid: "7136de9ce16f4703a928d1429952cdea" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 45057 } }
I20260811 09:07:51.677496 32563 catalog_manager.cc:5495] T ffb68a8b60e74c15b2b45b7cbca7e0e8 P 7136de9ce16f4703a928d1429952cdea reported cstate change: term changed from 0 to 1, leader changed from <none> to 7136de9ce16f4703a928d1429952cdea (127.28.154.131). New cstate: current_term: 1 leader_uuid: "7136de9ce16f4703a928d1429952cdea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b66124a6ca6244748b38ebb07c6ca40e" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8b65b15973c4d7d896a96567f97caee" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 38009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7136de9ce16f4703a928d1429952cdea" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 45057 } health_report { overall_health: HEALTHY } } }
I20260811 09:07:51.716930 1286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:51.711469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37242 (local address 127.28.154.131:45057)
0811 09:07:51.711654 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:51.711660 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:51.711683 (+ 23us) server_negotiation.cc:407] Connection header received
0811 09:07:51.711842 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:51.711848 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:51.711946 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:51.712052 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:07:51.712402 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:51.713576 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:51.714917 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:51.715601 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:51.715741 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:51.716129 (+ 388us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:07:51.716265 (+ 136us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:07:51.716399 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:51.716502 (+ 103us) server_negotiation.cc:299] Negotiation successful
0811 09:07:51.716591 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 09:07:51.726213 409 raft_consensus.cc:1261] T ffb68a8b60e74c15b2b45b7cbca7e0e8 P b66124a6ca6244748b38ebb07c6ca40e [term 1 FOLLOWER]: Refusing update from remote peer 7136de9ce16f4703a928d1429952cdea: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:07:51.726773 1316 consensus_queue.cc:1035] T ffb68a8b60e74c15b2b45b7cbca7e0e8 P 7136de9ce16f4703a928d1429952cdea [LEADER]: Connected to new peer: Peer: permanent_uuid: "b66124a6ca6244748b38ebb07c6ca40e" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:07:51.727012 685 raft_consensus.cc:1261] T ffb68a8b60e74c15b2b45b7cbca7e0e8 P e8b65b15973c4d7d896a96567f97caee [term 1 FOLLOWER]: Refusing update from remote peer 7136de9ce16f4703a928d1429952cdea: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:07:51.727708 1316 consensus_queue.cc:1035] T ffb68a8b60e74c15b2b45b7cbca7e0e8 P 7136de9ce16f4703a928d1429952cdea [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8b65b15973c4d7d896a96567f97caee" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 38009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 09:07:51.750286 1212 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260811 09:07:53.341593 32566 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47770:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:07:53.341858 32566 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47770:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:07:53.346627 32566 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 49c59055f30d4f5e8fc4a7bfebbdd512: Sending DeleteTablet for 3 replicas of tablet ffb68a8b60e74c15b2b45b7cbca7e0e8
I20260811 09:07:53.347620 361 tablet_service.cc:1514] Processing DeleteTablet for tablet ffb68a8b60e74c15b2b45b7cbca7e0e8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:07:53 UTC) from {username='kudu', principal='kudu/127.28.154.190@KRBTEST.COM'} at 127.0.0.1:49252
I20260811 09:07:53.347671 641 tablet_service.cc:1514] Processing DeleteTablet for tablet ffb68a8b60e74c15b2b45b7cbca7e0e8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:07:53 UTC) from {username='kudu', principal='kudu/127.28.154.190@KRBTEST.COM'} at 127.0.0.1:59830
I20260811 09:07:53.348317 910 tablet_service.cc:1514] Processing DeleteTablet for tablet ffb68a8b60e74c15b2b45b7cbca7e0e8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:07:53 UTC) from {username='kudu', principal='kudu/127.28.154.190@KRBTEST.COM'} at 127.0.0.1:37208
I20260811 09:07:53.348479 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 32707
I20260811 09:07:53.348686 1381 tablet_replica.cc:330] T ffb68a8b60e74c15b2b45b7cbca7e0e8 P e8b65b15973c4d7d896a96567f97caee: stopping tablet replica
I20260811 09:07:53.348937 1381 raft_consensus.cc:2229] T ffb68a8b60e74c15b2b45b7cbca7e0e8 P e8b65b15973c4d7d896a96567f97caee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:07:53.349320 1381 raft_consensus.cc:2258] T ffb68a8b60e74c15b2b45b7cbca7e0e8 P e8b65b15973c4d7d896a96567f97caee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:07:53.350778 1381 ts_tablet_manager.cc:1905] T ffb68a8b60e74c15b2b45b7cbca7e0e8 P e8b65b15973c4d7d896a96567f97caee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:07:53.355757 1384 tablet_replica.cc:330] T ffb68a8b60e74c15b2b45b7cbca7e0e8 P b66124a6ca6244748b38ebb07c6ca40e: stopping tablet replica
I20260811 09:07:53.356472 1384 raft_consensus.cc:2229] T ffb68a8b60e74c15b2b45b7cbca7e0e8 P b66124a6ca6244748b38ebb07c6ca40e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:07:53.357170 1384 raft_consensus.cc:2258] T ffb68a8b60e74c15b2b45b7cbca7e0e8 P b66124a6ca6244748b38ebb07c6ca40e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:07:53.358055 1387 tablet_replica.cc:330] T ffb68a8b60e74c15b2b45b7cbca7e0e8 P 7136de9ce16f4703a928d1429952cdea: stopping tablet replica
I20260811 09:07:53.359010 1384 ts_tablet_manager.cc:1905] T ffb68a8b60e74c15b2b45b7cbca7e0e8 P b66124a6ca6244748b38ebb07c6ca40e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:07:53.359212 1387 raft_consensus.cc:2229] T ffb68a8b60e74c15b2b45b7cbca7e0e8 P 7136de9ce16f4703a928d1429952cdea [term 1 LEADER]: Raft consensus shutting down.
I20260811 09:07:53.359712 1381 ts_tablet_manager.cc:1918] T ffb68a8b60e74c15b2b45b7cbca7e0e8 P e8b65b15973c4d7d896a96567f97caee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 09:07:53.359740 1387 raft_consensus.cc:2258] T ffb68a8b60e74c15b2b45b7cbca7e0e8 P 7136de9ce16f4703a928d1429952cdea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:07:53.359954 1381 log.cc:1196] T ffb68a8b60e74c15b2b45b7cbca7e0e8 P e8b65b15973c4d7d896a96567f97caee: Deleting WAL directory at /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439250192320-29290-0/minicluster-data/ts-1/wal/wals/ffb68a8b60e74c15b2b45b7cbca7e0e8
I20260811 09:07:53.360705 1381 ts_tablet_manager.cc:1939] T ffb68a8b60e74c15b2b45b7cbca7e0e8 P e8b65b15973c4d7d896a96567f97caee: Deleting consensus metadata
I20260811 09:07:53.361402 1387 ts_tablet_manager.cc:1905] T ffb68a8b60e74c15b2b45b7cbca7e0e8 P 7136de9ce16f4703a928d1429952cdea: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:07:53.362244 32534 catalog_manager.cc:4841] TS e8b65b15973c4d7d896a96567f97caee (127.28.154.130:38009): tablet ffb68a8b60e74c15b2b45b7cbca7e0e8 (table test-table [id=75dffa0a5561436ebb8a0956d33fe177]) successfully deleted
I20260811 09:07:53.366384 1384 ts_tablet_manager.cc:1918] T ffb68a8b60e74c15b2b45b7cbca7e0e8 P b66124a6ca6244748b38ebb07c6ca40e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 09:07:53.366556 1384 log.cc:1196] T ffb68a8b60e74c15b2b45b7cbca7e0e8 P b66124a6ca6244748b38ebb07c6ca40e: Deleting WAL directory at /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439250192320-29290-0/minicluster-data/ts-0/wal/wals/ffb68a8b60e74c15b2b45b7cbca7e0e8
I20260811 09:07:53.367100 1384 ts_tablet_manager.cc:1939] T ffb68a8b60e74c15b2b45b7cbca7e0e8 P b66124a6ca6244748b38ebb07c6ca40e: Deleting consensus metadata
I20260811 09:07:53.367739 1387 ts_tablet_manager.cc:1918] T ffb68a8b60e74c15b2b45b7cbca7e0e8 P 7136de9ce16f4703a928d1429952cdea: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 09:07:53.367879 1387 log.cc:1196] T ffb68a8b60e74c15b2b45b7cbca7e0e8 P 7136de9ce16f4703a928d1429952cdea: Deleting WAL directory at /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786439250192320-29290-0/minicluster-data/ts-2/wal/wals/ffb68a8b60e74c15b2b45b7cbca7e0e8
I20260811 09:07:53.368366 1387 ts_tablet_manager.cc:1939] T ffb68a8b60e74c15b2b45b7cbca7e0e8 P 7136de9ce16f4703a928d1429952cdea: Deleting consensus metadata
I20260811 09:07:53.368510 32534 catalog_manager.cc:4841] TS b66124a6ca6244748b38ebb07c6ca40e (127.28.154.129:41515): tablet ffb68a8b60e74c15b2b45b7cbca7e0e8 (table test-table [id=75dffa0a5561436ebb8a0956d33fe177]) successfully deleted
I20260811 09:07:53.369573 32534 catalog_manager.cc:4841] TS 7136de9ce16f4703a928d1429952cdea (127.28.154.131:45057): tablet ffb68a8b60e74c15b2b45b7cbca7e0e8 (table test-table [id=75dffa0a5561436ebb8a0956d33fe177]) successfully deleted
Aug 11 09:07:53 dist-test-slave-xt4k krb5kdc[32478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439271, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.28.154.129@KRBTEST.COM
I20260811 09:07:53.386564 1390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:53.350254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49258 (local address 127.28.154.129:41515)
0811 09:07:53.363249 (+ 12995us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:53.363256 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:53.363283 (+ 27us) server_negotiation.cc:407] Connection header received
0811 09:07:53.363344 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:53.363349 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:53.363465 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:53.363617 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:53.365454 (+ 1837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:53.366654 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:53.368728 (+ 2074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:53.369373 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:53.369528 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:53.379755 (+ 10227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:07:53.379781 (+ 26us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:07:53.379809 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:07:53.379854 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:07:53.383286 (+ 3432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:53.384093 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:53.384101 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:53.384104 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:53.384260 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:53.384691 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:53.384695 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:53.384698 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:53.385147 (+ 449us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:07:53.385529 (+ 382us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:53.385892 (+ 363us) server_negotiation.cc:299] Negotiation successful
0811 09:07:53.386156 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12811,"thread_start_us":12602,"threads_started":1}
W20260811 09:07:53.387291 499 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49258
W20260811 09:07:53.388041 29290 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 32707: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 09:07:53.388298 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 32707
I20260811 09:07:53.442873 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 517
Aug 11 09:07:53 dist-test-slave-xt4k krb5kdc[32478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439271, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.28.154.130@KRBTEST.COM
I20260811 09:07:53.459589 1392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:53.443840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59842 (local address 127.28.154.130:38009)
0811 09:07:53.444426 (+ 586us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:53.444431 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:53.444448 (+ 17us) server_negotiation.cc:407] Connection header received
0811 09:07:53.444505 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:53.444509 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:53.444571 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:53.444665 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:53.446067 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:53.447014 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:53.448572 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:53.449005 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:53.449111 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:53.453624 (+ 4513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:07:53.453643 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:07:53.453662 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:07:53.453703 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:07:53.456792 (+ 3089us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:53.457494 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:53.457503 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:53.457505 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:53.457635 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:53.458029 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:53.458033 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:53.458035 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:53.458420 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:07:53.458747 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:53.459014 (+ 267us) server_negotiation.cc:299] Negotiation successful
0811 09:07:53.459243 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":432,"thread_start_us":293,"threads_started":1}
W20260811 09:07:53.460330 753 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59842
W20260811 09:07:53.461063 29290 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 517: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 09:07:53.461215 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 517
I20260811 09:07:53.489817 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 768
Aug 11 09:07:53 dist-test-slave-xt4k krb5kdc[32478](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439271, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.28.154.131@KRBTEST.COM
I20260811 09:07:53.505733 1395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:53.490596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37254 (local address 127.28.154.131:45057)
0811 09:07:53.491107 (+ 511us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:53.491112 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:53.491127 (+ 15us) server_negotiation.cc:407] Connection header received
0811 09:07:53.491190 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:53.491194 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:53.491252 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:53.491369 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:53.492703 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:53.493467 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:53.494719 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:53.495171 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:53.495297 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:53.499652 (+ 4355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:07:53.499676 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:07:53.499699 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:07:53.499737 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:07:53.502814 (+ 3077us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:53.503547 (+ 733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:53.503556 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:53.503558 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:53.503700 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:53.504048 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:53.504052 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:53.504054 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:53.504591 (+ 537us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:07:53.504841 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:53.505212 (+ 371us) server_negotiation.cc:299] Negotiation successful
0811 09:07:53.505409 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":410,"thread_start_us":235,"threads_started":1}
W20260811 09:07:53.506562 1040 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37254
W20260811 09:07:53.507267 29290 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 768: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 09:07:53.507438 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 768
I20260811 09:07:53.543660 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 32496
I20260811 09:07:53.545086 32653 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:07:53.667477 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 32496
2026-08-11T09:07:53Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (8801 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439250192320-29290-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:07:53 dist-test-slave-xt4k krb5kdc[1405](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:07:53 dist-test-slave-xt4k krb5kdc[1405](info): set up 2 sockets
Aug 11 09:07:53 dist-test-slave-xt4k krb5kdc[1405](info): commencing operation
krb5kdc: starting...
W20260811 09:07:54.956712 29290 mini_kdc.cc:121] Time spent starting KDC: real 1.235s user 0.002s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439250192320-29290-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439250192320-29290-0/krb5kdc/test-user.keytab.
Aug 11 09:07:55 dist-test-slave-xt4k krb5kdc[1405](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439275, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:07:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:07:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.154.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.154.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.154.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab.
Entry for principal kudu/127.28.154.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab.
I20260811 09:07:55.188869 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439250192320-29290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439250192320-29290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439250192320-29290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.154.190:33013
--webserver_interface=127.28.154.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab
--principal=kudu/127.28.154.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:44147
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.154.190:33013
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:07:55.552618 1426 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:07:55.552942 1426 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:07:55.553052 1426 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:07:55.560472 1426 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:07:55.560595 1426 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:07:55.560673 1426 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:07:55.560750 1426 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:07:55.560798 1426 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:07:55.571153 1426 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:44147
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439250192320-29290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439250192320-29290-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.154.190:33013
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab
--principal=kudu/127.28.154.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.154.190:33013
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.154.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439250192320-29290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:07:55.572615 1426 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:07:55.574038 1426 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:07:55.586882 1438 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:07:55.586802 1434 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:07:55.586848 1435 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:07:55.587386 1426 server_base.cc:1034] running on GCE node
I20260811 09:07:55.588285 1426 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:07:55.589844 1426 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:07:55.591092 1426 hybrid_clock.cc:648] HybridClock initialized: now 1786439275591033 us; error 49 us; skew 500 ppm
I20260811 09:07:55.591439 1426 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:07:55 dist-test-slave-xt4k krb5kdc[1405](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439275, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:07:55.594817 1426 init.cc:377] Logged in from keytab as kudu/127.28.154.190@KRBTEST.COM (short username kudu)
I20260811 09:07:55.596495 1426 webserver.cc:460] Webserver started at http://127.28.154.190:33057/ using document root <none> and password file <none>
I20260811 09:07:55.597270 1426 fs_manager.cc:359] Metadata directory not provided
I20260811 09:07:55.597385 1426 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:07:55.597726 1426 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:07:55.600883 1426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439250192320-29290-0/minicluster-data/master-0/data/instance:
uuid: "484c96c1f25a4d4fbb62f33778ee2a0a"
format_stamp: "Formatted at 2026-08-11 09:07:55 on dist-test-slave-xt4k"
server_key: "3879ed837e93ce4306e2060a573abfec"
server_key_iv: "0d7e078691f3c0d9678a6412c1efc9a5"
server_key_version: "encryptionkey@0"
I20260811 09:07:55.601670 1426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439250192320-29290-0/minicluster-data/master-0/wal/instance:
uuid: "484c96c1f25a4d4fbb62f33778ee2a0a"
format_stamp: "Formatted at 2026-08-11 09:07:55 on dist-test-slave-xt4k"
server_key: "3879ed837e93ce4306e2060a573abfec"
server_key_iv: "0d7e078691f3c0d9678a6412c1efc9a5"
server_key_version: "encryptionkey@0"
I20260811 09:07:55.607502 1426 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260811 09:07:55.611334 1450 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:07:55.612885 1426 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 09:07:55.613078 1426 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439250192320-29290-0/minicluster-data/master-0/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439250192320-29290-0/minicluster-data/master-0/wal
uuid: "484c96c1f25a4d4fbb62f33778ee2a0a"
format_stamp: "Formatted at 2026-08-11 09:07:55 on dist-test-slave-xt4k"
server_key: "3879ed837e93ce4306e2060a573abfec"
server_key_iv: "0d7e078691f3c0d9678a6412c1efc9a5"
server_key_version: "encryptionkey@0"
I20260811 09:07:55.613301 1426 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439250192320-29290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439250192320-29290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439250192320-29290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:07:55.631161 1426 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:07:55.635236 1426 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:07:55.635512 1426 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:07:55.653204 1426 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.190:33013
I20260811 09:07:55.653235 1556 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.190:33013 every 8 connection(s)
I20260811 09:07:55.654547 1426 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
I20260811 09:07:55.658593 1558 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:07:55.661949 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 1426
I20260811 09:07:55.662192 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439250192320-29290-0/minicluster-data/master-0/wal/instance
I20260811 09:07:55.662987 29290 external_mini_cluster.cc:1428] Setting key 1253c7a954b9e4692cc82c207d1095c6
I20260811 09:07:55.671892 1558 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 484c96c1f25a4d4fbb62f33778ee2a0a: Bootstrap starting.
I20260811 09:07:55.675760 1558 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 484c96c1f25a4d4fbb62f33778ee2a0a: Neither blocks nor log segments found. Creating new log.
I20260811 09:07:55.677172 1558 log.cc:824] T 00000000000000000000000000000000 P 484c96c1f25a4d4fbb62f33778ee2a0a: Log is configured to *not* fsync() on all Append() calls
Aug 11 09:07:55 dist-test-slave-xt4k krb5kdc[1405](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439275, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.154.190@KRBTEST.COM
I20260811 09:07:55.680549 1558 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 484c96c1f25a4d4fbb62f33778ee2a0a: No bootstrap required, opened a new log
I20260811 09:07:55.687258 1561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:55.665160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39234 (local address 127.28.154.190:33013)
0811 09:07:55.665929 (+ 769us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:55.665940 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:55.665983 (+ 43us) server_negotiation.cc:407] Connection header received
0811 09:07:55.667642 (+ 1659us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:55.667666 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:55.668208 (+ 542us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:55.668650 (+ 442us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:55.670450 (+ 1800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:55.671789 (+ 1339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:55.673520 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:55.674214 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:55.674343 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:55.679148 (+ 4805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:07:55.679157 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:07:55.679174 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:07:55.679211 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:07:55.682582 (+ 3371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:55.683554 (+ 972us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:55.683562 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:55.683570 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:55.683786 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:55.684215 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:55.684220 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:55.684222 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:55.684891 (+ 669us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:07:55.685153 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:55.685672 (+ 519us) server_negotiation.cc:299] Negotiation successful
0811 09:07:55.686131 (+ 459us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":561,"thread_start_us":351,"threads_started":1}
I20260811 09:07:55.689764 1558 raft_consensus.cc:348] T 00000000000000000000000000000000 P 484c96c1f25a4d4fbb62f33778ee2a0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "484c96c1f25a4d4fbb62f33778ee2a0a" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 33013 } }
I20260811 09:07:55.690158 1558 raft_consensus.cc:374] T 00000000000000000000000000000000 P 484c96c1f25a4d4fbb62f33778ee2a0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:07:55.690259 1558 raft_consensus.cc:729] T 00000000000000000000000000000000 P 484c96c1f25a4d4fbb62f33778ee2a0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 484c96c1f25a4d4fbb62f33778ee2a0a, State: Initialized, Role: FOLLOWER
I20260811 09:07:55.690867 1558 consensus_queue.cc:260] T 00000000000000000000000000000000 P 484c96c1f25a4d4fbb62f33778ee2a0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "484c96c1f25a4d4fbb62f33778ee2a0a" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 33013 } }
I20260811 09:07:55.691075 1558 raft_consensus.cc:388] T 00000000000000000000000000000000 P 484c96c1f25a4d4fbb62f33778ee2a0a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:07:55.691174 1558 raft_consensus.cc:482] T 00000000000000000000000000000000 P 484c96c1f25a4d4fbb62f33778ee2a0a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:07:55.691288 1558 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 484c96c1f25a4d4fbb62f33778ee2a0a [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:07:55.693521 1558 raft_consensus.cc:504] T 00000000000000000000000000000000 P 484c96c1f25a4d4fbb62f33778ee2a0a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "484c96c1f25a4d4fbb62f33778ee2a0a" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 33013 } }
I20260811 09:07:55.694024 1558 leader_election.cc:302] T 00000000000000000000000000000000 P 484c96c1f25a4d4fbb62f33778ee2a0a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 484c96c1f25a4d4fbb62f33778ee2a0a; no voters:
I20260811 09:07:55.694552 1558 leader_election.cc:288] T 00000000000000000000000000000000 P 484c96c1f25a4d4fbb62f33778ee2a0a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:07:55.694717 1563 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 484c96c1f25a4d4fbb62f33778ee2a0a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:07:55.695405 1563 raft_consensus.cc:686] T 00000000000000000000000000000000 P 484c96c1f25a4d4fbb62f33778ee2a0a [term 1 LEADER]: Becoming Leader. State: Replica: 484c96c1f25a4d4fbb62f33778ee2a0a, State: Running, Role: LEADER
I20260811 09:07:55.696112 1563 consensus_queue.cc:237] T 00000000000000000000000000000000 P 484c96c1f25a4d4fbb62f33778ee2a0a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "484c96c1f25a4d4fbb62f33778ee2a0a" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 33013 } }
I20260811 09:07:55.696624 1558 sys_catalog.cc:564] T 00000000000000000000000000000000 P 484c96c1f25a4d4fbb62f33778ee2a0a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:07:55.700026 1565 sys_catalog.cc:455] T 00000000000000000000000000000000 P 484c96c1f25a4d4fbb62f33778ee2a0a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "484c96c1f25a4d4fbb62f33778ee2a0a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "484c96c1f25a4d4fbb62f33778ee2a0a" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 33013 } } }
I20260811 09:07:55.700042 1567 sys_catalog.cc:455] T 00000000000000000000000000000000 P 484c96c1f25a4d4fbb62f33778ee2a0a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 484c96c1f25a4d4fbb62f33778ee2a0a. Latest consensus state: current_term: 1 leader_uuid: "484c96c1f25a4d4fbb62f33778ee2a0a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "484c96c1f25a4d4fbb62f33778ee2a0a" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 33013 } } }
I20260811 09:07:55.700557 1565 sys_catalog.cc:458] T 00000000000000000000000000000000 P 484c96c1f25a4d4fbb62f33778ee2a0a [sys.catalog]: This master's current role is: LEADER
I20260811 09:07:55.700557 1567 sys_catalog.cc:458] T 00000000000000000000000000000000 P 484c96c1f25a4d4fbb62f33778ee2a0a [sys.catalog]: This master's current role is: LEADER
I20260811 09:07:55.703939 1581 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:07:55.709713 1581 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:07:55.719005 1581 catalog_manager.cc:1347] Generated new cluster ID: 94e35c91e61b4b209230399bfc2f66d0
I20260811 09:07:55.719141 1581 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:07:55.736330 1581 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:07:55.737777 1581 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:07:55.746524 1581 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 484c96c1f25a4d4fbb62f33778ee2a0a: Generated new TSK 0
I20260811 09:07:55.747424 1581 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.28.154.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.154.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.154.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.129.keytab.
Entry for principal kudu/127.28.154.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.129.keytab.
I20260811 09:07:55.813478 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439250192320-29290-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439250192320-29290-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439250192320-29290-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439250192320-29290-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.154.129:0
--local_ip_for_outbound_sockets=127.28.154.129
--webserver_interface=127.28.154.129
--webserver_port=0
--tserver_master_addrs=127.28.154.190:33013
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.129.keytab
--principal=kudu/127.28.154.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:44147
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:07:56.171231 1605 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:07:56.171552 1605 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:07:56.171654 1605 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:07:56.179040 1605 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:07:56.179164 1605 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:07:56.179280 1605 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.154.129
I20260811 09:07:56.189481 1605 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:44147
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439250192320-29290-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439250192320-29290-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.129.keytab
--principal=kudu/127.28.154.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.154.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439250192320-29290-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.154.129
--webserver_port=0
--tserver_master_addrs=127.28.154.190:33013
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.154.129
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439250192320-29290-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:07:56.190776 1605 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:07:56.192071 1605 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:07:56.205739 1614 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:07:56.205788 1620 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:07:56.205817 1613 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:07:56.206655 1605 server_base.cc:1034] running on GCE node
I20260811 09:07:56.207342 1605 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:07:56.208470 1605 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:07:56.209753 1605 hybrid_clock.cc:648] HybridClock initialized: now 1786439276209633 us; error 50 us; skew 500 ppm
I20260811 09:07:56.210176 1605 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:07:56 dist-test-slave-xt4k krb5kdc[1405](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439276, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:07:56.214246 1605 init.cc:377] Logged in from keytab as kudu/127.28.154.129@KRBTEST.COM (short username kudu)
I20260811 09:07:56.215839 1605 webserver.cc:460] Webserver started at http://127.28.154.129:45591/ using document root <none> and password file <none>
I20260811 09:07:56.216676 1605 fs_manager.cc:359] Metadata directory not provided
I20260811 09:07:56.217000 1605 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:07:56.217303 1605 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:07:56.220523 1605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439250192320-29290-0/minicluster-data/ts-0/data/instance:
uuid: "b7aa916edfe54b828ee0297558ec091d"
format_stamp: "Formatted at 2026-08-11 09:07:56 on dist-test-slave-xt4k"
server_key: "8ac8009884ac355d1af05d1439a350b4"
server_key_iv: "3f94def6f44aa4d68f7ab3f234cf0b04"
server_key_version: "encryptionkey@0"
I20260811 09:07:56.221292 1605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439250192320-29290-0/minicluster-data/ts-0/wal/instance:
uuid: "b7aa916edfe54b828ee0297558ec091d"
format_stamp: "Formatted at 2026-08-11 09:07:56 on dist-test-slave-xt4k"
server_key: "8ac8009884ac355d1af05d1439a350b4"
server_key_iv: "3f94def6f44aa4d68f7ab3f234cf0b04"
server_key_version: "encryptionkey@0"
I20260811 09:07:56.226789 1605 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260811 09:07:56.230799 1634 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:07:56.232407 1605 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 09:07:56.232602 1605 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439250192320-29290-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439250192320-29290-0/minicluster-data/ts-0/wal
uuid: "b7aa916edfe54b828ee0297558ec091d"
format_stamp: "Formatted at 2026-08-11 09:07:56 on dist-test-slave-xt4k"
server_key: "8ac8009884ac355d1af05d1439a350b4"
server_key_iv: "3f94def6f44aa4d68f7ab3f234cf0b04"
server_key_version: "encryptionkey@0"
I20260811 09:07:56.232813 1605 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439250192320-29290-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439250192320-29290-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439250192320-29290-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:07:56.246383 1605 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:07:56.252702 1605 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:07:56.253059 1605 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:07:56.254254 1605 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:07:56.256080 1605 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:07:56.256174 1605 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:07:56.256294 1605 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:07:56.256364 1605 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:07:56.290220 1605 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.129:35033
I20260811 09:07:56.290242 1854 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.129:35033 every 8 connection(s)
I20260811 09:07:56.291539 1605 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439250192320-29290-0/minicluster-data/ts-0/data/info.pb
I20260811 09:07:56.299202 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 1605
I20260811 09:07:56.299420 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439250192320-29290-0/minicluster-data/ts-0/wal/instance
I20260811 09:07:56.300076 29290 external_mini_cluster.cc:1428] Setting key a0e22ab2ae861f7730da773e13897a9e
Aug 11 09:07:56 dist-test-slave-xt4k krb5kdc[1405](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439276, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.129@KRBTEST.COM for kudu/127.28.154.190@KRBTEST.COM
I20260811 09:07:56.313884 1860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:56.294231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.129:47899 (local address 127.28.154.190:33013)
0811 09:07:56.294682 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:56.294687 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:56.295626 (+ 939us) server_negotiation.cc:407] Connection header received
0811 09:07:56.297515 (+ 1889us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:56.297521 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:56.297603 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:56.297702 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:56.299324 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:56.300114 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:56.302290 (+ 2176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:56.302942 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:56.303083 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:56.306622 (+ 3539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:07:56.306628 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:07:56.306633 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:07:56.306671 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:07:56.309775 (+ 3104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:56.310780 (+ 1005us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:56.310785 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:56.310788 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:56.310923 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:56.311417 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:56.311421 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:56.311423 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:56.311713 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:07:56.311828 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:56.312779 (+ 951us) server_negotiation.cc:299] Negotiation successful
0811 09:07:56.312996 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":226,"threads_started":1}
I20260811 09:07:56.314800 1861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:56.294742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:33013 (local address 127.28.154.129:47899)
0811 09:07:56.295446 (+ 704us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:56.295505 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 09:07:56.297247 (+ 1742us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:07:56.297944 (+ 697us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:07:56.297953 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:07:56.298461 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:07:56.299142 (+ 681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:56.299167 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:56.300310 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:56.300316 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:56.302117 (+ 1801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:56.302135 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:56.303186 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:56.303191 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:56.303412 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:56.304132 (+ 720us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:07:56.304157 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:07:56.306388 (+ 2231us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:07:56.309946 (+ 3558us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:07:56.309956 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:07:56.309999 (+ 43us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:07:56.310637 (+ 638us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:07:56.311042 (+ 405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:07:56.311046 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:07:56.311050 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:07:56.311297 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:07:56.311834 (+ 537us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:07:56.311842 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:07:56.312316 (+ 474us) client_negotiation.cc:769] Sending connection context
0811 09:07:56.312656 (+ 340us) client_negotiation.cc:240] Negotiation successful
0811 09:07:56.313368 (+ 712us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":252,"threads_started":1}
I20260811 09:07:56.316601 1855 heartbeater.cc:347] Connected to a master server at 127.28.154.190:33013
I20260811 09:07:56.317098 1855 heartbeater.cc:464] Registering TS with master...
I20260811 09:07:56.317907 1855 heartbeater.cc:511] Master 127.28.154.190:33013 requested a full tablet report, sending...
I20260811 09:07:56.320686 1485 ts_manager.cc:194] Registered new tserver with Master: b7aa916edfe54b828ee0297558ec091d (127.28.154.129:35033)
I20260811 09:07:56.322886 1485 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.154.129@KRBTEST.COM'} at 127.28.154.129:47899
WARNING: no policy specified for kudu/127.28.154.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.154.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.154.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.130.keytab.
Entry for principal kudu/127.28.154.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.130.keytab.
W20260811 09:07:57.243407 29290 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.28.154.130: real 0.941s user 0.004s sys 0.011s
I20260811 09:07:57.244118 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439250192320-29290-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439250192320-29290-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439250192320-29290-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439250192320-29290-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.154.130:0
--local_ip_for_outbound_sockets=127.28.154.130
--webserver_interface=127.28.154.130
--webserver_port=0
--tserver_master_addrs=127.28.154.190:33013
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.130.keytab
--principal=kudu/127.28.154.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:44147
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 09:07:57.326402 1855 heartbeater.cc:503] Master 127.28.154.190:33013 was elected leader, sending a full tablet report...
W20260811 09:07:57.597581 1864 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:07:57.597935 1864 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:07:57.598075 1864 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:07:57.605890 1864 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:07:57.606045 1864 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:07:57.606175 1864 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.154.130
I20260811 09:07:57.617700 1864 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:44147
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439250192320-29290-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439250192320-29290-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.130.keytab
--principal=kudu/127.28.154.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.154.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439250192320-29290-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.154.130
--webserver_port=0
--tserver_master_addrs=127.28.154.190:33013
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.154.130
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439250192320-29290-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:07:57.619093 1864 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:07:57.620437 1864 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:07:57.634701 1871 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:07:57.634724 1876 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:07:57.634716 1872 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:07:57.635303 1864 server_base.cc:1034] running on GCE node
I20260811 09:07:57.636272 1864 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:07:57.637432 1864 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:07:57.638715 1864 hybrid_clock.cc:648] HybridClock initialized: now 1786439277638642 us; error 57 us; skew 500 ppm
I20260811 09:07:57.639063 1864 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:07:57 dist-test-slave-xt4k krb5kdc[1405](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439277, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:07:57.642515 1864 init.cc:377] Logged in from keytab as kudu/127.28.154.130@KRBTEST.COM (short username kudu)
I20260811 09:07:57.644053 1864 webserver.cc:460] Webserver started at http://127.28.154.130:43511/ using document root <none> and password file <none>
I20260811 09:07:57.644851 1864 fs_manager.cc:359] Metadata directory not provided
I20260811 09:07:57.644981 1864 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:07:57.645303 1864 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:07:57.648416 1864 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439250192320-29290-0/minicluster-data/ts-1/data/instance:
uuid: "c4b4f527e00b49fd8795d3c8bb5a78d6"
format_stamp: "Formatted at 2026-08-11 09:07:57 on dist-test-slave-xt4k"
server_key: "82fd2d5eb6ea8b119eea8f94ee9804c8"
server_key_iv: "261df11260f0c9d67c1cf762fcea6144"
server_key_version: "encryptionkey@0"
I20260811 09:07:57.649228 1864 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439250192320-29290-0/minicluster-data/ts-1/wal/instance:
uuid: "c4b4f527e00b49fd8795d3c8bb5a78d6"
format_stamp: "Formatted at 2026-08-11 09:07:57 on dist-test-slave-xt4k"
server_key: "82fd2d5eb6ea8b119eea8f94ee9804c8"
server_key_iv: "261df11260f0c9d67c1cf762fcea6144"
server_key_version: "encryptionkey@0"
I20260811 09:07:57.655198 1864 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260811 09:07:57.659829 1892 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:07:57.661446 1864 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 09:07:57.661600 1864 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439250192320-29290-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439250192320-29290-0/minicluster-data/ts-1/wal
uuid: "c4b4f527e00b49fd8795d3c8bb5a78d6"
format_stamp: "Formatted at 2026-08-11 09:07:57 on dist-test-slave-xt4k"
server_key: "82fd2d5eb6ea8b119eea8f94ee9804c8"
server_key_iv: "261df11260f0c9d67c1cf762fcea6144"
server_key_version: "encryptionkey@0"
I20260811 09:07:57.661792 1864 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439250192320-29290-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439250192320-29290-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439250192320-29290-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:07:57.685472 1864 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:07:57.691316 1864 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:07:57.691668 1864 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:07:57.692998 1864 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:07:57.694826 1864 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:07:57.694926 1864 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:07:57.695055 1864 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:07:57.695115 1864 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:07:57.733403 1864 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.130:34329
I20260811 09:07:57.733439 2100 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.130:34329 every 8 connection(s)
I20260811 09:07:57.735236 1864 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439250192320-29290-0/minicluster-data/ts-1/data/info.pb
I20260811 09:07:57.737164 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 1864
I20260811 09:07:57.737337 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439250192320-29290-0/minicluster-data/ts-1/wal/instance
I20260811 09:07:57.737946 29290 external_mini_cluster.cc:1428] Setting key a8d707749cc0a13bb4c0a5bec4b22ee2
WARNING: no policy specified for kudu/127.28.154.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.154.131@KRBTEST.COM" created.
Aug 11 09:07:57 dist-test-slave-xt4k krb5kdc[1405](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439277, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.130@KRBTEST.COM for kudu/127.28.154.190@KRBTEST.COM
I20260811 09:07:57.777336 2109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:57.738453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.130:53885 (local address 127.28.154.190:33013)
0811 09:07:57.756834 (+ 18381us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:57.756842 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:57.757627 (+ 785us) server_negotiation.cc:407] Connection header received
0811 09:07:57.759715 (+ 2088us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:57.759722 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:57.759815 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:57.759914 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:57.761645 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:57.762488 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:57.764114 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:57.764605 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:57.764723 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:57.770178 (+ 5455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:07:57.770188 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:07:57.770195 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:07:57.770249 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:07:57.773811 (+ 3562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:57.774791 (+ 980us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:57.774798 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:57.774802 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:57.774962 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:57.775471 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:57.775475 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:57.775477 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:57.775769 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:07:57.775908 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:57.776690 (+ 782us) server_negotiation.cc:299] Negotiation successful
0811 09:07:57.776944 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18205,"thread_start_us":18065,"threads_started":1}
I20260811 09:07:57.778384 2111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:57.739194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:33013 (local address 127.28.154.130:53885)
0811 09:07:57.757409 (+ 18215us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:57.757477 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0811 09:07:57.759385 (+ 1908us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:07:57.760207 (+ 822us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:07:57.760216 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:07:57.760823 (+ 607us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:07:57.761441 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:57.761460 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:57.762660 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:57.762664 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:57.763954 (+ 1290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:57.763970 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:57.764849 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:57.764856 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:57.765174 (+ 318us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:57.765846 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:07:57.765875 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:07:57.769793 (+ 3918us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:07:57.774013 (+ 4220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:07:57.774025 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:07:57.774044 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:07:57.774640 (+ 596us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:07:57.775078 (+ 438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:07:57.775082 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:07:57.775085 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:07:57.775332 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:07:57.775900 (+ 568us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:07:57.775909 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:07:57.776278 (+ 369us) client_negotiation.cc:769] Sending connection context
0811 09:07:57.776577 (+ 299us) client_negotiation.cc:240] Negotiation successful
0811 09:07:57.777166 (+ 589us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17864,"thread_start_us":17607,"threads_started":1}
I20260811 09:07:57.779925 2102 heartbeater.cc:347] Connected to a master server at 127.28.154.190:33013
I20260811 09:07:57.780226 2102 heartbeater.cc:464] Registering TS with master...
I20260811 09:07:57.780978 2102 heartbeater.cc:511] Master 127.28.154.190:33013 requested a full tablet report, sending...
I20260811 09:07:57.782415 1485 ts_manager.cc:194] Registered new tserver with Master: c4b4f527e00b49fd8795d3c8bb5a78d6 (127.28.154.130:34329)
I20260811 09:07:57.783215 1485 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.154.130@KRBTEST.COM'} at 127.28.154.130:53885
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.154.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.131.keytab.
Entry for principal kudu/127.28.154.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.131.keytab.
I20260811 09:07:57.800673 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439250192320-29290-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439250192320-29290-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439250192320-29290-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439250192320-29290-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.154.131:0
--local_ip_for_outbound_sockets=127.28.154.131
--webserver_interface=127.28.154.131
--webserver_port=0
--tserver_master_addrs=127.28.154.190:33013
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.131.keytab
--principal=kudu/127.28.154.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:44147
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:07:58.161792 2120 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:07:58.162113 2120 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:07:58.162209 2120 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:07:58.169778 2120 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:07:58.169922 2120 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:07:58.170178 2120 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.154.131
I20260811 09:07:58.181317 2120 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:44147
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439250192320-29290-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439250192320-29290-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.131.keytab
--principal=kudu/127.28.154.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.154.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439250192320-29290-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.154.131
--webserver_port=0
--tserver_master_addrs=127.28.154.190:33013
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.154.131
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439250192320-29290-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:07:58.182745 2120 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:07:58.184191 2120 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:07:58.197890 2141 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:07:58.197966 2135 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:07:58.198208 2134 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:07:58.199296 2120 server_base.cc:1034] running on GCE node
I20260811 09:07:58.200047 2120 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:07:58.201128 2120 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:07:58.202366 2120 hybrid_clock.cc:648] HybridClock initialized: now 1786439278202311 us; error 33 us; skew 500 ppm
I20260811 09:07:58.202754 2120 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:07:58 dist-test-slave-xt4k krb5kdc[1405](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439278, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:07:58.206363 2120 init.cc:377] Logged in from keytab as kudu/127.28.154.131@KRBTEST.COM (short username kudu)
I20260811 09:07:58.208114 2120 webserver.cc:460] Webserver started at http://127.28.154.131:44427/ using document root <none> and password file <none>
I20260811 09:07:58.208897 2120 fs_manager.cc:359] Metadata directory not provided
I20260811 09:07:58.209023 2120 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:07:58.209326 2120 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:07:58.212369 2120 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439250192320-29290-0/minicluster-data/ts-2/data/instance:
uuid: "a918b79ea41c46ee9faa98b82eda1fde"
format_stamp: "Formatted at 2026-08-11 09:07:58 on dist-test-slave-xt4k"
server_key: "2dc8c8941d516b3252667f43e330ff4b"
server_key_iv: "fc9c4d2f505ff5126bdd3ef4289aabb9"
server_key_version: "encryptionkey@0"
I20260811 09:07:58.213187 2120 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439250192320-29290-0/minicluster-data/ts-2/wal/instance:
uuid: "a918b79ea41c46ee9faa98b82eda1fde"
format_stamp: "Formatted at 2026-08-11 09:07:58 on dist-test-slave-xt4k"
server_key: "2dc8c8941d516b3252667f43e330ff4b"
server_key_iv: "fc9c4d2f505ff5126bdd3ef4289aabb9"
server_key_version: "encryptionkey@0"
I20260811 09:07:58.218803 2120 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 09:07:58.222447 2153 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:07:58.223948 2120 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 09:07:58.224155 2120 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439250192320-29290-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439250192320-29290-0/minicluster-data/ts-2/wal
uuid: "a918b79ea41c46ee9faa98b82eda1fde"
format_stamp: "Formatted at 2026-08-11 09:07:58 on dist-test-slave-xt4k"
server_key: "2dc8c8941d516b3252667f43e330ff4b"
server_key_iv: "fc9c4d2f505ff5126bdd3ef4289aabb9"
server_key_version: "encryptionkey@0"
I20260811 09:07:58.224416 2120 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439250192320-29290-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439250192320-29290-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439250192320-29290-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:07:58.252811 2120 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:07:58.257436 2120 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:07:58.257743 2120 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:07:58.258807 2120 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:07:58.260263 2120 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:07:58.260368 2120 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:07:58.260509 2120 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:07:58.260560 2120 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:07:58.296572 2120 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.131:39781
I20260811 09:07:58.296598 2364 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.131:39781 every 8 connection(s)
I20260811 09:07:58.297981 2120 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439250192320-29290-0/minicluster-data/ts-2/data/info.pb
I20260811 09:07:58.298512 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 2120
I20260811 09:07:58.298774 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786439250192320-29290-0/minicluster-data/ts-2/wal/instance
I20260811 09:07:58.299440 29290 external_mini_cluster.cc:1428] Setting key 07e2e2be377b4118784c5569c91ad561
Aug 11 09:07:58 dist-test-slave-xt4k krb5kdc[1405](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439278, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.131@KRBTEST.COM for kudu/127.28.154.190@KRBTEST.COM
I20260811 09:07:58.320209 2372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:58.300845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.131:43943 (local address 127.28.154.190:33013)
0811 09:07:58.301303 (+ 458us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:58.301309 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:58.302531 (+ 1222us) server_negotiation.cc:407] Connection header received
0811 09:07:58.304731 (+ 2200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:58.304736 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:58.304813 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:58.304917 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:58.306500 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:58.307342 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:58.308992 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:07:58.309429 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:07:58.309578 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:58.313338 (+ 3760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:07:58.313344 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:07:58.313348 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:07:58.313396 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:07:58.316493 (+ 3097us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:58.317378 (+ 885us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:58.317383 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:58.317386 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:58.317537 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:07:58.318158 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:07:58.318162 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:07:58.318165 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:07:58.318672 (+ 507us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:07:58.318832 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0811 09:07:58.319728 (+ 896us) server_negotiation.cc:299] Negotiation successful
0811 09:07:58.319917 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":258,"threads_started":1}
I20260811 09:07:58.321228 2374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:07:58.301470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:33013 (local address 127.28.154.131:43943)
0811 09:07:58.302344 (+ 874us) negotiation.cc:107] Waiting for socket to connect
0811 09:07:58.302409 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0811 09:07:58.304452 (+ 2043us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:07:58.305123 (+ 671us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:07:58.305131 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:07:58.305694 (+ 563us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:07:58.306301 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:58.306319 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:58.307479 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:58.307483 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:58.308841 (+ 1358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:07:58.308861 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:07:58.309669 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:07:58.309674 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:07:58.309888 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:07:58.310545 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:07:58.310571 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:07:58.313067 (+ 2496us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:07:58.316639 (+ 3572us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:07:58.316648 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:07:58.316666 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:07:58.317204 (+ 538us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:07:58.317679 (+ 475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:07:58.317683 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:07:58.317685 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:07:58.317939 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:07:58.318891 (+ 952us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:07:58.318918 (+ 27us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:07:58.319318 (+ 400us) client_negotiation.cc:769] Sending connection context
0811 09:07:58.319590 (+ 272us) client_negotiation.cc:240] Negotiation successful
0811 09:07:58.320052 (+ 462us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":582,"thread_start_us":344,"threads_started":1}
I20260811 09:07:58.322834 2366 heartbeater.cc:347] Connected to a master server at 127.28.154.190:33013
I20260811 09:07:58.323139 2366 heartbeater.cc:464] Registering TS with master...
I20260811 09:07:58.323839 2366 heartbeater.cc:511] Master 127.28.154.190:33013 requested a full tablet report, sending...
I20260811 09:07:58.325340 1485 ts_manager.cc:194] Registered new tserver with Master: a918b79ea41c46ee9faa98b82eda1fde (127.28.154.131:39781)
I20260811 09:07:58.326138 1485 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.154.131@KRBTEST.COM'} at 127.28.154.131:43943
I20260811 09:07:58.328657 29290 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 09:07:58.367166 2372 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:07:58.364076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39236 (local address 127.28.154.190:33013)
0811 09:07:58.364284 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0811 09:07:58.364291 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:07:58.364845 (+ 554us) server_negotiation.cc:407] Connection header received
0811 09:07:58.365036 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:07:58.365051 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:07:58.365156 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:07:58.365253 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:07:58.366902 (+ 1649us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39236: BlockingRecv error: recv got EOF from 127.0.0.1:39236 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 09:07:58.370615 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 1605
W20260811 09:07:58.373540 29290 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 1605: Not authorized: Client connection negotiation failed: client connection to 127.28.154.129:35033: server requires authentication, but client does not have Kerberos credentials available
I20260811 09:07:58.373775 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 1605
I20260811 09:07:58.400056 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 1864
W20260811 09:07:58.402931 29290 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 1864: Not authorized: Client connection negotiation failed: client connection to 127.28.154.130:34329: server requires authentication, but client does not have Kerberos credentials available
I20260811 09:07:58.403069 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 1864
I20260811 09:07:58.421258 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 2120
W20260811 09:07:58.424481 29290 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 2120: Not authorized: Client connection negotiation failed: client connection to 127.28.154.131:39781: server requires authentication, but client does not have Kerberos credentials available
I20260811 09:07:58.424654 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 2120
I20260811 09:07:58.445122 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 1426
I20260811 09:07:58.446327 1544 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:07:58.550555 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 1426
2026-08-11T09:07:58Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (4880 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786439250192320-29290-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:07:58 dist-test-slave-xt4k krb5kdc[2404](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:07:58 dist-test-slave-xt4k krb5kdc[2404](info): set up 2 sockets
Aug 11 09:07:58 dist-test-slave-xt4k krb5kdc[2404](info): commencing operation
krb5kdc: starting...
W20260811 09:07:59.847426 29290 mini_kdc.cc:121] Time spent starting KDC: real 1.246s user 0.004s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786439250192320-29290-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786439250192320-29290-0/krb5kdc/test-user.keytab.
Aug 11 09:07:59 dist-test-slave-xt4k krb5kdc[2404](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439279, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:08:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:08:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.154.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.154.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.154.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab.
Entry for principal kudu/127.28.154.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab.
I20260811 09:08:00.077415 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786439250192320-29290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786439250192320-29290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786439250192320-29290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.154.190:37017
--webserver_interface=127.28.154.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab
--principal=kudu/127.28.154.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:46629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.154.190:37017
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:08:00.413723 2433 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:08:00.414052 2433 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:08:00.414187 2433 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:08:00.421762 2433 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:08:00.421916 2433 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:08:00.422001 2433 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:08:00.422083 2433 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:08:00.422149 2433 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:08:00.432516 2433 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:46629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786439250192320-29290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786439250192320-29290-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.154.190:37017
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab
--principal=kudu/127.28.154.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.154.190:37017
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.154.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786439250192320-29290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:08:00.434207 2433 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:08:00.435675 2433 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:08:00.448949 2439 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:08:00.448987 2440 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:08:00.448922 2447 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:08:00.450152 2433 server_base.cc:1034] running on GCE node
I20260811 09:08:00.451081 2433 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:08:00.452720 2433 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:08:00.454121 2433 hybrid_clock.cc:648] HybridClock initialized: now 1786439280454052 us; error 55 us; skew 500 ppm
I20260811 09:08:00.454542 2433 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:08:00 dist-test-slave-xt4k krb5kdc[2404](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439280, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:08:00.457924 2433 init.cc:377] Logged in from keytab as kudu/127.28.154.190@KRBTEST.COM (short username kudu)
I20260811 09:08:00.459853 2433 webserver.cc:460] Webserver started at http://127.28.154.190:37693/ using document root <none> and password file <none>
I20260811 09:08:00.460690 2433 fs_manager.cc:359] Metadata directory not provided
I20260811 09:08:00.460808 2433 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:08:00.461159 2433 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:08:00.466536 2433 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786439250192320-29290-0/minicluster-data/master-0/data/instance:
uuid: "5c6d865b49f146aab5c8e26c96d9d33b"
format_stamp: "Formatted at 2026-08-11 09:08:00 on dist-test-slave-xt4k"
server_key: "6a302eadc4a46034335811d13ca7c3b0"
server_key_iv: "4cbd52144eee597b51048808ecbb5b07"
server_key_version: "encryptionkey@0"
I20260811 09:08:00.468369 2433 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786439250192320-29290-0/minicluster-data/master-0/wal/instance:
uuid: "5c6d865b49f146aab5c8e26c96d9d33b"
format_stamp: "Formatted at 2026-08-11 09:08:00 on dist-test-slave-xt4k"
server_key: "6a302eadc4a46034335811d13ca7c3b0"
server_key_iv: "4cbd52144eee597b51048808ecbb5b07"
server_key_version: "encryptionkey@0"
I20260811 09:08:00.474548 2433 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260811 09:08:00.479195 2463 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:08:00.482064 2433 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260811 09:08:00.482293 2433 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786439250192320-29290-0/minicluster-data/master-0/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786439250192320-29290-0/minicluster-data/master-0/wal
uuid: "5c6d865b49f146aab5c8e26c96d9d33b"
format_stamp: "Formatted at 2026-08-11 09:08:00 on dist-test-slave-xt4k"
server_key: "6a302eadc4a46034335811d13ca7c3b0"
server_key_iv: "4cbd52144eee597b51048808ecbb5b07"
server_key_version: "encryptionkey@0"
I20260811 09:08:00.482555 2433 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786439250192320-29290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786439250192320-29290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786439250192320-29290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:08:00.499997 2433 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:08:00.506013 2433 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:08:00.506320 2433 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:08:00.526028 2433 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.190:37017
I20260811 09:08:00.526075 2568 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.190:37017 every 8 connection(s)
I20260811 09:08:00.527439 2433 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
I20260811 09:08:00.530854 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 2433
I20260811 09:08:00.531111 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786439250192320-29290-0/minicluster-data/master-0/wal/instance
I20260811 09:08:00.531754 29290 external_mini_cluster.cc:1428] Setting key 401a0487ee8e4a1e19723bfb168de99a
I20260811 09:08:00.531843 2570 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:08:00.546558 2570 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c6d865b49f146aab5c8e26c96d9d33b: Bootstrap starting.
Aug 11 09:08:00 dist-test-slave-xt4k krb5kdc[2404](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439279, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.154.190@KRBTEST.COM
I20260811 09:08:00.550740 2570 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5c6d865b49f146aab5c8e26c96d9d33b: Neither blocks nor log segments found. Creating new log.
I20260811 09:08:00.552148 2570 log.cc:824] T 00000000000000000000000000000000 P 5c6d865b49f146aab5c8e26c96d9d33b: Log is configured to *not* fsync() on all Append() calls
I20260811 09:08:00.555761 2570 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c6d865b49f146aab5c8e26c96d9d33b: No bootstrap required, opened a new log
I20260811 09:08:00.557024 2576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:00.533595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52744 (local address 127.28.154.190:37017)
0811 09:08:00.534547 (+ 952us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:00.534563 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:00.534614 (+ 51us) server_negotiation.cc:407] Connection header received
0811 09:08:00.536991 (+ 2377us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:00.537026 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:00.537618 (+ 592us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:00.538130 (+ 512us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:00.539485 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:00.541440 (+ 1955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:00.542916 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:00.544009 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:00.544203 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:00.548571 (+ 4368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:00.548584 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:00.548604 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:08:00.548660 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:00.552512 (+ 3852us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:00.553271 (+ 759us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:00.553280 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:00.553289 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:00.553509 (+ 220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:00.554008 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:00.554012 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:00.554015 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:00.554639 (+ 624us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:00.554890 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:00.555418 (+ 528us) server_negotiation.cc:299] Negotiation successful
0811 09:08:00.555886 (+ 468us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":649,"thread_start_us":389,"threads_started":1}
I20260811 09:08:00.561441 29290 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 09:08:00.564806 2570 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5c6d865b49f146aab5c8e26c96d9d33b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c6d865b49f146aab5c8e26c96d9d33b" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 37017 } }
I20260811 09:08:00.565232 2570 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5c6d865b49f146aab5c8e26c96d9d33b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:08:00.565335 2570 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5c6d865b49f146aab5c8e26c96d9d33b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c6d865b49f146aab5c8e26c96d9d33b, State: Initialized, Role: FOLLOWER
I20260811 09:08:00.566087 2570 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5c6d865b49f146aab5c8e26c96d9d33b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c6d865b49f146aab5c8e26c96d9d33b" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 37017 } }
I20260811 09:08:00.566340 2570 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5c6d865b49f146aab5c8e26c96d9d33b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:08:00.566476 2570 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5c6d865b49f146aab5c8e26c96d9d33b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:08:00.566646 2570 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5c6d865b49f146aab5c8e26c96d9d33b [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:08:00.569160 2570 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5c6d865b49f146aab5c8e26c96d9d33b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c6d865b49f146aab5c8e26c96d9d33b" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 37017 } }
I20260811 09:08:00.569700 2570 leader_election.cc:302] T 00000000000000000000000000000000 P 5c6d865b49f146aab5c8e26c96d9d33b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5c6d865b49f146aab5c8e26c96d9d33b; no voters:
I20260811 09:08:00.570318 2570 leader_election.cc:288] T 00000000000000000000000000000000 P 5c6d865b49f146aab5c8e26c96d9d33b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:08:00.570473 2583 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5c6d865b49f146aab5c8e26c96d9d33b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:08:00.571488 2583 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5c6d865b49f146aab5c8e26c96d9d33b [term 1 LEADER]: Becoming Leader. State: Replica: 5c6d865b49f146aab5c8e26c96d9d33b, State: Running, Role: LEADER
I20260811 09:08:00.572134 2583 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5c6d865b49f146aab5c8e26c96d9d33b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c6d865b49f146aab5c8e26c96d9d33b" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 37017 } }
I20260811 09:08:00.572588 2570 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5c6d865b49f146aab5c8e26c96d9d33b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:08:00.578218 2586 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c6d865b49f146aab5c8e26c96d9d33b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5c6d865b49f146aab5c8e26c96d9d33b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c6d865b49f146aab5c8e26c96d9d33b" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 37017 } } }
I20260811 09:08:00.578801 2586 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c6d865b49f146aab5c8e26c96d9d33b [sys.catalog]: This master's current role is: LEADER
I20260811 09:08:00.579324 2587 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c6d865b49f146aab5c8e26c96d9d33b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5c6d865b49f146aab5c8e26c96d9d33b. Latest consensus state: current_term: 1 leader_uuid: "5c6d865b49f146aab5c8e26c96d9d33b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c6d865b49f146aab5c8e26c96d9d33b" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 37017 } } }
I20260811 09:08:00.579653 2587 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c6d865b49f146aab5c8e26c96d9d33b [sys.catalog]: This master's current role is: LEADER
I20260811 09:08:00.582237 2602 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:08:00.589969 2602 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:08:00.601245 2602 catalog_manager.cc:1347] Generated new cluster ID: 3a84ba37dd9d4e2f9a81052beb5f481f
I20260811 09:08:00.601392 2602 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:08:00.613235 2576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:00.605686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52758 (local address 127.28.154.190:37017)
0811 09:08:00.605858 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:00.605865 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:00.606380 (+ 515us) server_negotiation.cc:407] Connection header received
0811 09:08:00.606563 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:00.606570 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:00.606643 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:00.606729 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:00.607764 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:00.608604 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:00.609850 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:00.610332 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:00.610449 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:00.611508 (+ 1059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:00.611513 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:00.611517 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:08:00.612433 (+ 916us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:00.612487 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:08:00.612527 (+ 40us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:00.612627 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:00.612764 (+ 137us) server_negotiation.cc:299] Negotiation successful
0811 09:08:00.612873 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":22,"server-negotiator.queue_time_us":39}
I20260811 09:08:00.622224 2602 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:08:00.623577 2602 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:08:00.635576 2602 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5c6d865b49f146aab5c8e26c96d9d33b: Generated new TSK 0
I20260811 09:08:00.636762 2602 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 09:08:00.651526 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 2433
I20260811 09:08:00.652660 2561 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:08:00.744453 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 2433
2026-08-11T09:08:00Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (2184 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439250192320-29290-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:08:00 dist-test-slave-xt4k krb5kdc[2650](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:08:00 dist-test-slave-xt4k krb5kdc[2650](info): set up 2 sockets
Aug 11 09:08:00 dist-test-slave-xt4k krb5kdc[2650](info): commencing operation
krb5kdc: starting...
W20260811 09:08:02.002686 29290 mini_kdc.cc:121] Time spent starting KDC: real 1.216s user 0.000s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439250192320-29290-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439250192320-29290-0/krb5kdc/test-user.keytab.
Aug 11 09:08:02 dist-test-slave-xt4k krb5kdc[2650](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439282, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:08:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:08:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.154.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.154.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.154.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab.
Entry for principal kudu/127.28.154.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab.
I20260811 09:08:02.232846 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439250192320-29290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439250192320-29290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439250192320-29290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.154.190:33881
--webserver_interface=127.28.154.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab
--principal=kudu/127.28.154.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:44313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.154.190:33881
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:08:02.569698 2673 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:08:02.570065 2673 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:08:02.570205 2673 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:08:02.577816 2673 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:08:02.577970 2673 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:08:02.578055 2673 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:08:02.578135 2673 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:08:02.578193 2673 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:08:02.588526 2673 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:44313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439250192320-29290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439250192320-29290-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.154.190:33881
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab
--principal=kudu/127.28.154.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.154.190:33881
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.154.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439250192320-29290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:08:02.589937 2673 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:08:02.591235 2673 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:08:02.603952 2683 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:08:02.604017 2681 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:08:02.604163 2680 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:08:02.604869 2673 server_base.cc:1034] running on GCE node
I20260811 09:08:02.605741 2673 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:08:02.607240 2673 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:08:02.608484 2673 hybrid_clock.cc:648] HybridClock initialized: now 1786439282608405 us; error 56 us; skew 500 ppm
I20260811 09:08:02.608844 2673 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:08:02 dist-test-slave-xt4k krb5kdc[2650](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439282, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:08:02.612169 2673 init.cc:377] Logged in from keytab as kudu/127.28.154.190@KRBTEST.COM (short username kudu)
I20260811 09:08:02.613886 2673 webserver.cc:460] Webserver started at http://127.28.154.190:40467/ using document root <none> and password file <none>
I20260811 09:08:02.614588 2673 fs_manager.cc:359] Metadata directory not provided
I20260811 09:08:02.614696 2673 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:08:02.614964 2673 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:08:02.618072 2673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439250192320-29290-0/minicluster-data/master-0/data/instance:
uuid: "34b34e7c0d584be58c58493a6c9b5a26"
format_stamp: "Formatted at 2026-08-11 09:08:02 on dist-test-slave-xt4k"
server_key: "977ce5b2e098513f30cd2cf0f42a8164"
server_key_iv: "bd725f76bf29383a85cb846fbf7dc303"
server_key_version: "encryptionkey@0"
I20260811 09:08:02.618822 2673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439250192320-29290-0/minicluster-data/master-0/wal/instance:
uuid: "34b34e7c0d584be58c58493a6c9b5a26"
format_stamp: "Formatted at 2026-08-11 09:08:02 on dist-test-slave-xt4k"
server_key: "977ce5b2e098513f30cd2cf0f42a8164"
server_key_iv: "bd725f76bf29383a85cb846fbf7dc303"
server_key_version: "encryptionkey@0"
I20260811 09:08:02.624380 2673 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 09:08:02.627947 2699 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:08:02.629515 2673 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 09:08:02.629701 2673 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439250192320-29290-0/minicluster-data/master-0/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439250192320-29290-0/minicluster-data/master-0/wal
uuid: "34b34e7c0d584be58c58493a6c9b5a26"
format_stamp: "Formatted at 2026-08-11 09:08:02 on dist-test-slave-xt4k"
server_key: "977ce5b2e098513f30cd2cf0f42a8164"
server_key_iv: "bd725f76bf29383a85cb846fbf7dc303"
server_key_version: "encryptionkey@0"
I20260811 09:08:02.629906 2673 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439250192320-29290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439250192320-29290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439250192320-29290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:08:02.661253 2673 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:08:02.665616 2673 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:08:02.665894 2673 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:08:02.683487 2673 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.190:33881
I20260811 09:08:02.683503 2799 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.190:33881 every 8 connection(s)
I20260811 09:08:02.684912 2673 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
I20260811 09:08:02.686043 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 2673
I20260811 09:08:02.686254 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439250192320-29290-0/minicluster-data/master-0/wal/instance
I20260811 09:08:02.686846 29290 external_mini_cluster.cc:1428] Setting key bd56cf98cab27b151ae706dade00ab4e
I20260811 09:08:02.689213 2800 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 09:08:02 dist-test-slave-xt4k krb5kdc[2650](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439282, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.154.190@KRBTEST.COM
I20260811 09:08:02.702199 2800 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34b34e7c0d584be58c58493a6c9b5a26: Bootstrap starting.
I20260811 09:08:02.705513 2800 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 34b34e7c0d584be58c58493a6c9b5a26: Neither blocks nor log segments found. Creating new log.
I20260811 09:08:02.706715 2800 log.cc:824] T 00000000000000000000000000000000 P 34b34e7c0d584be58c58493a6c9b5a26: Log is configured to *not* fsync() on all Append() calls
I20260811 09:08:02.709658 2800 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34b34e7c0d584be58c58493a6c9b5a26: No bootstrap required, opened a new log
I20260811 09:08:02.710248 2812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:02.688911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47240 (local address 127.28.154.190:33881)
0811 09:08:02.689713 (+ 802us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:02.689728 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:02.689779 (+ 51us) server_negotiation.cc:407] Connection header received
0811 09:08:02.691953 (+ 2174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:02.691982 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:02.692525 (+ 543us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:02.692968 (+ 443us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:02.694259 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:02.696304 (+ 2045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:02.697557 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:02.698591 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:02.698779 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:02.702870 (+ 4091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:02.702879 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:02.702895 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:08:02.702935 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:02.705947 (+ 3012us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:02.706830 (+ 883us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:02.706841 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:02.706850 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:02.707069 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:02.707537 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:02.707541 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:02.707543 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:02.708086 (+ 543us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:02.708393 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:02.708852 (+ 459us) server_negotiation.cc:299] Negotiation successful
0811 09:08:02.709260 (+ 408us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":564,"thread_start_us":322,"threads_started":1}
I20260811 09:08:02.718268 2800 raft_consensus.cc:348] T 00000000000000000000000000000000 P 34b34e7c0d584be58c58493a6c9b5a26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34b34e7c0d584be58c58493a6c9b5a26" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 33881 } }
I20260811 09:08:02.718672 2800 raft_consensus.cc:374] T 00000000000000000000000000000000 P 34b34e7c0d584be58c58493a6c9b5a26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:08:02.718789 2800 raft_consensus.cc:729] T 00000000000000000000000000000000 P 34b34e7c0d584be58c58493a6c9b5a26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34b34e7c0d584be58c58493a6c9b5a26, State: Initialized, Role: FOLLOWER
I20260811 09:08:02.719373 2800 consensus_queue.cc:260] T 00000000000000000000000000000000 P 34b34e7c0d584be58c58493a6c9b5a26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34b34e7c0d584be58c58493a6c9b5a26" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 33881 } }
I20260811 09:08:02.719595 2800 raft_consensus.cc:388] T 00000000000000000000000000000000 P 34b34e7c0d584be58c58493a6c9b5a26 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:08:02.719725 2800 raft_consensus.cc:482] T 00000000000000000000000000000000 P 34b34e7c0d584be58c58493a6c9b5a26 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:08:02.719879 2800 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 34b34e7c0d584be58c58493a6c9b5a26 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:08:02.722054 2800 raft_consensus.cc:504] T 00000000000000000000000000000000 P 34b34e7c0d584be58c58493a6c9b5a26 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34b34e7c0d584be58c58493a6c9b5a26" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 33881 } }
I20260811 09:08:02.722555 2800 leader_election.cc:302] T 00000000000000000000000000000000 P 34b34e7c0d584be58c58493a6c9b5a26 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 34b34e7c0d584be58c58493a6c9b5a26; no voters:
I20260811 09:08:02.723117 2800 leader_election.cc:288] T 00000000000000000000000000000000 P 34b34e7c0d584be58c58493a6c9b5a26 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:08:02.723245 2818 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 34b34e7c0d584be58c58493a6c9b5a26 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:08:02.724310 2818 raft_consensus.cc:686] T 00000000000000000000000000000000 P 34b34e7c0d584be58c58493a6c9b5a26 [term 1 LEADER]: Becoming Leader. State: Replica: 34b34e7c0d584be58c58493a6c9b5a26, State: Running, Role: LEADER
I20260811 09:08:02.724869 2818 consensus_queue.cc:237] T 00000000000000000000000000000000 P 34b34e7c0d584be58c58493a6c9b5a26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34b34e7c0d584be58c58493a6c9b5a26" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 33881 } }
I20260811 09:08:02.725344 2800 sys_catalog.cc:564] T 00000000000000000000000000000000 P 34b34e7c0d584be58c58493a6c9b5a26 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:08:02.727662 2825 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34b34e7c0d584be58c58493a6c9b5a26 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 34b34e7c0d584be58c58493a6c9b5a26. Latest consensus state: current_term: 1 leader_uuid: "34b34e7c0d584be58c58493a6c9b5a26" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34b34e7c0d584be58c58493a6c9b5a26" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 33881 } } }
I20260811 09:08:02.727627 2819 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34b34e7c0d584be58c58493a6c9b5a26 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "34b34e7c0d584be58c58493a6c9b5a26" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34b34e7c0d584be58c58493a6c9b5a26" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 33881 } } }
I20260811 09:08:02.727977 2825 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34b34e7c0d584be58c58493a6c9b5a26 [sys.catalog]: This master's current role is: LEADER
I20260811 09:08:02.728024 2819 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34b34e7c0d584be58c58493a6c9b5a26 [sys.catalog]: This master's current role is: LEADER
I20260811 09:08:02.730827 2834 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:08:02.735515 2834 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:08:02.746649 2834 catalog_manager.cc:1347] Generated new cluster ID: 8b808dce28d2458fa5d7d57e7c51f5a3
I20260811 09:08:02.746783 2834 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:08:02.762486 2834 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:08:02.763851 2834 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:08:02.772863 2834 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 34b34e7c0d584be58c58493a6c9b5a26: Generated new TSK 0
I20260811 09:08:02.773728 2834 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.28.154.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.154.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.154.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.129.keytab.
Entry for principal kudu/127.28.154.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.129.keytab.
I20260811 09:08:02.833972 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439250192320-29290-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439250192320-29290-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439250192320-29290-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439250192320-29290-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.154.129:0
--local_ip_for_outbound_sockets=127.28.154.129
--webserver_interface=127.28.154.129
--webserver_port=0
--tserver_master_addrs=127.28.154.190:33881
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.129.keytab
--principal=kudu/127.28.154.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:44313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:08:03.173470 2853 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:08:03.173804 2853 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:08:03.173946 2853 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:08:03.181490 2853 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:08:03.181631 2853 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:08:03.181797 2853 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.154.129
I20260811 09:08:03.193007 2853 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:44313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439250192320-29290-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439250192320-29290-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.129.keytab
--principal=kudu/127.28.154.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.154.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439250192320-29290-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.154.129
--webserver_port=0
--tserver_master_addrs=127.28.154.190:33881
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.154.129
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439250192320-29290-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:08:03.194453 2853 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:08:03.195839 2853 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:08:03.209753 2863 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:08:03.209792 2866 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:08:03.210142 2860 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:08:03.211027 2853 server_base.cc:1034] running on GCE node
I20260811 09:08:03.211884 2853 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:08:03.213593 2853 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:08:03.214833 2853 hybrid_clock.cc:648] HybridClock initialized: now 1786439283214774 us; error 54 us; skew 500 ppm
I20260811 09:08:03.215318 2853 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:08:03 dist-test-slave-xt4k krb5kdc[2650](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439283, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:08:03.218838 2853 init.cc:377] Logged in from keytab as kudu/127.28.154.129@KRBTEST.COM (short username kudu)
I20260811 09:08:03.220435 2853 webserver.cc:460] Webserver started at http://127.28.154.129:35855/ using document root <none> and password file <none>
I20260811 09:08:03.221186 2853 fs_manager.cc:359] Metadata directory not provided
I20260811 09:08:03.221302 2853 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:08:03.221613 2853 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:08:03.224797 2853 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439250192320-29290-0/minicluster-data/ts-0/data/instance:
uuid: "3fb1c75ae76f49af9ac300084c840b7e"
format_stamp: "Formatted at 2026-08-11 09:08:03 on dist-test-slave-xt4k"
server_key: "646312ef3bc30d679239fb5a5082eced"
server_key_iv: "105769e4c962df84e53b53026b9f7594"
server_key_version: "encryptionkey@0"
I20260811 09:08:03.225611 2853 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439250192320-29290-0/minicluster-data/ts-0/wal/instance:
uuid: "3fb1c75ae76f49af9ac300084c840b7e"
format_stamp: "Formatted at 2026-08-11 09:08:03 on dist-test-slave-xt4k"
server_key: "646312ef3bc30d679239fb5a5082eced"
server_key_iv: "105769e4c962df84e53b53026b9f7594"
server_key_version: "encryptionkey@0"
I20260811 09:08:03.231141 2853 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 09:08:03.234928 2880 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:08:03.236863 2853 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260811 09:08:03.237037 2853 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439250192320-29290-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439250192320-29290-0/minicluster-data/ts-0/wal
uuid: "3fb1c75ae76f49af9ac300084c840b7e"
format_stamp: "Formatted at 2026-08-11 09:08:03 on dist-test-slave-xt4k"
server_key: "646312ef3bc30d679239fb5a5082eced"
server_key_iv: "105769e4c962df84e53b53026b9f7594"
server_key_version: "encryptionkey@0"
I20260811 09:08:03.237241 2853 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439250192320-29290-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439250192320-29290-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439250192320-29290-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:08:03.254274 2853 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:08:03.258497 2853 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:08:03.258785 2853 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:08:03.259783 2853 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:08:03.261219 2853 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:08:03.261298 2853 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:08:03.261436 2853 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:08:03.261490 2853 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:08:03.292922 2853 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.129:37755
I20260811 09:08:03.292979 3097 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.129:37755 every 8 connection(s)
I20260811 09:08:03.294229 2853 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439250192320-29290-0/minicluster-data/ts-0/data/info.pb
I20260811 09:08:03.296926 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 2853
I20260811 09:08:03.297147 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439250192320-29290-0/minicluster-data/ts-0/wal/instance
I20260811 09:08:03.297896 29290 external_mini_cluster.cc:1428] Setting key 4e4938c511e9274db813d1707aa8c6c7
Aug 11 09:08:03 dist-test-slave-xt4k krb5kdc[2650](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439283, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.129@KRBTEST.COM for kudu/127.28.154.190@KRBTEST.COM
I20260811 09:08:03.314571 3105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:03.296850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.129:35801 (local address 127.28.154.190:33881)
0811 09:08:03.297459 (+ 609us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:03.297466 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:03.298413 (+ 947us) server_negotiation.cc:407] Connection header received
0811 09:08:03.300574 (+ 2161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:03.300580 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:03.300663 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:03.300758 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:03.302302 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:03.303035 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:03.304627 (+ 1592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:03.305038 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:03.305153 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:03.308843 (+ 3690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:03.308848 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:03.308852 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:08:03.308893 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:03.311416 (+ 2523us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:03.312155 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:03.312159 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:03.312161 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:03.312283 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:03.312735 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:03.312739 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:03.312741 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:03.313026 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:03.313162 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:03.313962 (+ 800us) server_negotiation.cc:299] Negotiation successful
0811 09:08:03.314203 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":425,"thread_start_us":297,"threads_started":1}
I20260811 09:08:03.315596 3106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:03.297415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:33881 (local address 127.28.154.129:35801)
0811 09:08:03.298217 (+ 802us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:03.298272 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:03.300258 (+ 1986us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:03.301013 (+ 755us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:03.301023 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:03.301554 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:08:03.302134 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:03.302152 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:03.303204 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:03.303208 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:03.304487 (+ 1279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:03.304501 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:03.305262 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:03.305270 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:03.305510 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:03.306156 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:08:03.306179 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:08:03.308619 (+ 2440us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:08:03.311568 (+ 2949us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:08:03.311579 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:08:03.311599 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:08:03.312028 (+ 429us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:08:03.312413 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:08:03.312418 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:08:03.312422 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:08:03.312619 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:08:03.313152 (+ 533us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:08:03.313160 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:08:03.313568 (+ 408us) client_negotiation.cc:769] Sending connection context
0811 09:08:03.313817 (+ 249us) client_negotiation.cc:240] Negotiation successful
0811 09:08:03.314393 (+ 576us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":520,"thread_start_us":316,"threads_started":1}
I20260811 09:08:03.317430 3101 heartbeater.cc:347] Connected to a master server at 127.28.154.190:33881
I20260811 09:08:03.317795 3101 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.28.154.130@KRBTEST.COM; defaulting to no policy
I20260811 09:08:03.318714 3101 heartbeater.cc:511] Master 127.28.154.190:33881 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.154.130@KRBTEST.COM" created.
I20260811 09:08:03.321205 2736 ts_manager.cc:194] Registered new tserver with Master: 3fb1c75ae76f49af9ac300084c840b7e (127.28.154.129:37755)
I20260811 09:08:03.322716 2736 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.154.129@KRBTEST.COM'} at 127.28.154.129:35801
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.154.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.130.keytab.
Entry for principal kudu/127.28.154.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.130.keytab.
I20260811 09:08:03.347591 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439250192320-29290-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439250192320-29290-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439250192320-29290-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439250192320-29290-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.154.130:0
--local_ip_for_outbound_sockets=127.28.154.130
--webserver_interface=127.28.154.130
--webserver_port=0
--tserver_master_addrs=127.28.154.190:33881
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.130.keytab
--principal=kudu/127.28.154.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:44313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:08:03.683053 3112 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:08:03.683362 3112 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:08:03.683488 3112 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:08:03.690940 3112 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:08:03.691064 3112 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:08:03.691211 3112 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.154.130
I20260811 09:08:03.701217 3112 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:44313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439250192320-29290-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439250192320-29290-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.130.keytab
--principal=kudu/127.28.154.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.154.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439250192320-29290-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.154.130
--webserver_port=0
--tserver_master_addrs=127.28.154.190:33881
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.154.130
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439250192320-29290-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:08:03.702576 3112 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:08:03.703915 3112 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:08:03.716718 3120 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:08:03.716763 3121 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:08:03.716650 3126 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:08:03.717618 3112 server_base.cc:1034] running on GCE node
I20260811 09:08:03.718236 3112 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:08:03.719161 3112 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:08:03.720392 3112 hybrid_clock.cc:648] HybridClock initialized: now 1786439283720315 us; error 46 us; skew 500 ppm
I20260811 09:08:03.720707 3112 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:08:03 dist-test-slave-xt4k krb5kdc[2650](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439283, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:08:03.726579 3112 init.cc:377] Logged in from keytab as kudu/127.28.154.130@KRBTEST.COM (short username kudu)
I20260811 09:08:03.728096 3112 webserver.cc:460] Webserver started at http://127.28.154.130:34637/ using document root <none> and password file <none>
I20260811 09:08:03.728837 3112 fs_manager.cc:359] Metadata directory not provided
I20260811 09:08:03.728960 3112 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:08:03.729282 3112 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:08:03.732375 3112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439250192320-29290-0/minicluster-data/ts-1/data/instance:
uuid: "0cc5b390156246f49f0a9cb0739661ca"
format_stamp: "Formatted at 2026-08-11 09:08:03 on dist-test-slave-xt4k"
server_key: "1fb8f7ab724dfdeb5aad17c8a32e4174"
server_key_iv: "794e94973c7b4ab8704cb7978fb5a57d"
server_key_version: "encryptionkey@0"
I20260811 09:08:03.733210 3112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439250192320-29290-0/minicluster-data/ts-1/wal/instance:
uuid: "0cc5b390156246f49f0a9cb0739661ca"
format_stamp: "Formatted at 2026-08-11 09:08:03 on dist-test-slave-xt4k"
server_key: "1fb8f7ab724dfdeb5aad17c8a32e4174"
server_key_iv: "794e94973c7b4ab8704cb7978fb5a57d"
server_key_version: "encryptionkey@0"
I20260811 09:08:03.738572 3112 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 09:08:03.742281 3137 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:08:03.743680 3112 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 09:08:03.743840 3112 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439250192320-29290-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439250192320-29290-0/minicluster-data/ts-1/wal
uuid: "0cc5b390156246f49f0a9cb0739661ca"
format_stamp: "Formatted at 2026-08-11 09:08:03 on dist-test-slave-xt4k"
server_key: "1fb8f7ab724dfdeb5aad17c8a32e4174"
server_key_iv: "794e94973c7b4ab8704cb7978fb5a57d"
server_key_version: "encryptionkey@0"
I20260811 09:08:03.744021 3112 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439250192320-29290-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439250192320-29290-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439250192320-29290-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:08:03.775132 3112 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:08:03.779078 3112 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:08:03.779347 3112 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:08:03.780287 3112 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:08:03.781607 3112 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:08:03.781687 3112 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:08:03.781803 3112 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:08:03.781864 3112 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:08:03.812712 3112 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.130:39153
I20260811 09:08:03.812764 3366 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.130:39153 every 8 connection(s)
I20260811 09:08:03.813956 3112 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439250192320-29290-0/minicluster-data/ts-1/data/info.pb
I20260811 09:08:03.820050 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 3112
I20260811 09:08:03.820233 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439250192320-29290-0/minicluster-data/ts-1/wal/instance
I20260811 09:08:03.820724 29290 external_mini_cluster.cc:1428] Setting key 3592dd815867d7c170873de289046b5e
Aug 11 09:08:03 dist-test-slave-xt4k krb5kdc[2650](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439283, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.130@KRBTEST.COM for kudu/127.28.154.190@KRBTEST.COM
I20260811 09:08:03.834277 3371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:03.816548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.130:40075 (local address 127.28.154.190:33881)
0811 09:08:03.816983 (+ 435us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:03.816987 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:03.817832 (+ 845us) server_negotiation.cc:407] Connection header received
0811 09:08:03.819610 (+ 1778us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:03.819616 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:03.819721 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:03.819814 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:03.821565 (+ 1751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:03.822440 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:03.824044 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:03.824493 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:03.824617 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:03.828004 (+ 3387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:03.828009 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:03.828012 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:08:03.828051 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:03.830985 (+ 2934us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:03.831881 (+ 896us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:03.831885 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:03.831887 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:03.832008 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:03.832487 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:03.832491 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:03.832493 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:03.832758 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:03.832865 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:03.833739 (+ 874us) server_negotiation.cc:299] Negotiation successful
0811 09:08:03.833901 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327,"thread_start_us":220,"threads_started":1}
I20260811 09:08:03.835553 3372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:03.817011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:33881 (local address 127.28.154.130:40075)
0811 09:08:03.817680 (+ 669us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:03.817725 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:03.819337 (+ 1612us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:03.820059 (+ 722us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:03.820067 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:03.820666 (+ 599us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:08:03.821345 (+ 679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:03.821373 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:03.822625 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:03.822630 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:03.823898 (+ 1268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:03.823911 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:03.824721 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:03.824728 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:03.824951 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:03.825613 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:08:03.825644 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:08:03.827792 (+ 2148us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:08:03.831134 (+ 3342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:08:03.831143 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:08:03.831159 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:08:03.831772 (+ 613us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:08:03.832107 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:08:03.832111 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:08:03.832114 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:08:03.832386 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:08:03.832882 (+ 496us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:08:03.832891 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:08:03.833361 (+ 470us) client_negotiation.cc:769] Sending connection context
0811 09:08:03.833649 (+ 288us) client_negotiation.cc:240] Negotiation successful
0811 09:08:03.834162 (+ 513us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":276,"threads_started":1}
I20260811 09:08:03.837102 3367 heartbeater.cc:347] Connected to a master server at 127.28.154.190:33881
I20260811 09:08:03.837388 3367 heartbeater.cc:464] Registering TS with master...
I20260811 09:08:03.838140 3367 heartbeater.cc:511] Master 127.28.154.190:33881 requested a full tablet report, sending...
I20260811 09:08:03.839638 2736 ts_manager.cc:194] Registered new tserver with Master: 0cc5b390156246f49f0a9cb0739661ca (127.28.154.130:39153)
I20260811 09:08:03.840466 2736 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.154.130@KRBTEST.COM'} at 127.28.154.130:40075
WARNING: no policy specified for kudu/127.28.154.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.154.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.154.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.131.keytab.
Entry for principal kudu/127.28.154.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.131.keytab.
I20260811 09:08:03.869249 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439250192320-29290-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439250192320-29290-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439250192320-29290-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439250192320-29290-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.154.131:0
--local_ip_for_outbound_sockets=127.28.154.131
--webserver_interface=127.28.154.131
--webserver_port=0
--tserver_master_addrs=127.28.154.190:33881
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.131.keytab
--principal=kudu/127.28.154.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:44313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:08:04.206169 3378 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:08:04.206451 3378 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:08:04.206585 3378 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:08:04.213778 3378 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:08:04.213899 3378 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:08:04.214054 3378 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.154.131
I20260811 09:08:04.223891 3378 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:44313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439250192320-29290-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439250192320-29290-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.131.keytab
--principal=kudu/127.28.154.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.154.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439250192320-29290-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.154.131
--webserver_port=0
--tserver_master_addrs=127.28.154.190:33881
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.154.131
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439250192320-29290-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:08:04.225275 3378 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:08:04.226711 3378 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:08:04.239671 3393 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:08:04.239749 3396 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:08:04.240000 3392 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:08:04.240621 3378 server_base.cc:1034] running on GCE node
I20260811 09:08:04.241240 3378 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:08:04.242280 3378 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:08:04.243531 3378 hybrid_clock.cc:648] HybridClock initialized: now 1786439284243464 us; error 58 us; skew 500 ppm
I20260811 09:08:04.243860 3378 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:08:04 dist-test-slave-xt4k krb5kdc[2650](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439284, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:08:04.246914 3378 init.cc:377] Logged in from keytab as kudu/127.28.154.131@KRBTEST.COM (short username kudu)
I20260811 09:08:04.248446 3378 webserver.cc:460] Webserver started at http://127.28.154.131:34393/ using document root <none> and password file <none>
I20260811 09:08:04.249114 3378 fs_manager.cc:359] Metadata directory not provided
I20260811 09:08:04.249213 3378 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:08:04.249462 3378 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:08:04.252245 3378 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439250192320-29290-0/minicluster-data/ts-2/data/instance:
uuid: "470398d25d2c4698a27aff74959df3c0"
format_stamp: "Formatted at 2026-08-11 09:08:04 on dist-test-slave-xt4k"
server_key: "929804cb428a9e12c911b094152a8021"
server_key_iv: "3514cbc6a36fc4f9c79404e6771c1f5f"
server_key_version: "encryptionkey@0"
I20260811 09:08:04.253026 3378 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439250192320-29290-0/minicluster-data/ts-2/wal/instance:
uuid: "470398d25d2c4698a27aff74959df3c0"
format_stamp: "Formatted at 2026-08-11 09:08:04 on dist-test-slave-xt4k"
server_key: "929804cb428a9e12c911b094152a8021"
server_key_iv: "3514cbc6a36fc4f9c79404e6771c1f5f"
server_key_version: "encryptionkey@0"
I20260811 09:08:04.257938 3378 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 09:08:04.261344 3405 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:08:04.262702 3378 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 09:08:04.262838 3378 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439250192320-29290-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439250192320-29290-0/minicluster-data/ts-2/wal
uuid: "470398d25d2c4698a27aff74959df3c0"
format_stamp: "Formatted at 2026-08-11 09:08:04 on dist-test-slave-xt4k"
server_key: "929804cb428a9e12c911b094152a8021"
server_key_iv: "3514cbc6a36fc4f9c79404e6771c1f5f"
server_key_version: "encryptionkey@0"
I20260811 09:08:04.263000 3378 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439250192320-29290-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439250192320-29290-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439250192320-29290-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:08:04.279886 3378 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:08:04.284462 3378 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:08:04.284765 3378 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:08:04.285805 3378 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:08:04.287235 3378 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:08:04.287309 3378 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:08:04.287415 3378 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:08:04.287459 3378 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:08:04.320835 3378 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.131:42577
I20260811 09:08:04.320889 3618 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.131:42577 every 8 connection(s)
I20260811 09:08:04.322118 3378 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439250192320-29290-0/minicluster-data/ts-2/data/info.pb
I20260811 09:08:04.326226 3101 heartbeater.cc:503] Master 127.28.154.190:33881 was elected leader, sending a full tablet report...
I20260811 09:08:04.330475 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 3378
I20260811 09:08:04.330664 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786439250192320-29290-0/minicluster-data/ts-2/wal/instance
I20260811 09:08:04.331202 29290 external_mini_cluster.cc:1428] Setting key b8b22ee168a0b438e33b9abe3f00aa0b
Aug 11 09:08:04 dist-test-slave-xt4k krb5kdc[2650](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439284, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.131@KRBTEST.COM for kudu/127.28.154.190@KRBTEST.COM
I20260811 09:08:04.343072 3371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:04.324657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.131:57915 (local address 127.28.154.190:33881)
0811 09:08:04.324875 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:04.324882 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:04.326428 (+ 1546us) server_negotiation.cc:407] Connection header received
0811 09:08:04.328323 (+ 1895us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:04.328329 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:04.328468 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:04.328558 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:04.330215 (+ 1657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:04.331179 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:04.332960 (+ 1781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:04.333403 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:04.333529 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:04.337226 (+ 3697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:04.337231 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:04.337235 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:08:04.337270 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:04.339852 (+ 2582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:04.340652 (+ 800us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:04.340657 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:04.340659 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:04.340809 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:04.341338 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:04.341342 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:04.341344 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:04.341693 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:04.341818 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:04.342639 (+ 821us) server_negotiation.cc:299] Negotiation successful
0811 09:08:04.342825 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 09:08:04.344075 3628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:04.325219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:33881 (local address 127.28.154.131:57915)
0811 09:08:04.326074 (+ 855us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:04.326117 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:04.328068 (+ 1951us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:04.328826 (+ 758us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:04.328836 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:04.329397 (+ 561us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:08:04.330051 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:04.330070 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:04.331335 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:04.331339 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:04.332827 (+ 1488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:04.332841 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:04.333630 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:04.333637 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:04.333888 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:04.334621 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:08:04.334646 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:08:04.337001 (+ 2355us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:08:04.339992 (+ 2991us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:08:04.340002 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:08:04.340021 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:08:04.340504 (+ 483us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:08:04.340962 (+ 458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:08:04.340967 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:08:04.340971 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:08:04.341198 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:08:04.341827 (+ 629us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:08:04.341833 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:08:04.342224 (+ 391us) client_negotiation.cc:769] Sending connection context
0811 09:08:04.342492 (+ 268us) client_negotiation.cc:240] Negotiation successful
0811 09:08:04.342922 (+ 430us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":563,"thread_start_us":298,"threads_started":1}
I20260811 09:08:04.345769 3622 heartbeater.cc:347] Connected to a master server at 127.28.154.190:33881
I20260811 09:08:04.346094 3622 heartbeater.cc:464] Registering TS with master...
I20260811 09:08:04.346923 3622 heartbeater.cc:511] Master 127.28.154.190:33881 requested a full tablet report, sending...
I20260811 09:08:04.348460 2736 ts_manager.cc:194] Registered new tserver with Master: 470398d25d2c4698a27aff74959df3c0 (127.28.154.131:42577)
I20260811 09:08:04.349277 2736 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.154.131@KRBTEST.COM'} at 127.28.154.131:57915
I20260811 09:08:04.359135 29290 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 09:08:04 dist-test-slave-xt4k krb5kdc[2650](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439284, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 11 09:08:04 dist-test-slave-xt4k krb5kdc[2650](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439284, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.28.154.190@KRBTEST.COM
I20260811 09:08:04.413383 3371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:04.398498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47252 (local address 127.28.154.190:33881)
0811 09:08:04.398752 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:04.398760 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:04.399074 (+ 314us) server_negotiation.cc:407] Connection header received
0811 09:08:04.399288 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:04.399292 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:04.399391 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:04.399516 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:04.400928 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:04.401777 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:04.403122 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:04.403575 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:04.403692 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:04.407772 (+ 4080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:04.407778 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:04.407781 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:08:04.407824 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:04.410556 (+ 2732us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:04.411300 (+ 744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:04.411306 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:04.411308 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:04.411464 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:04.411903 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:04.411909 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:04.411911 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:04.412278 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:04.412519 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:04.412818 (+ 299us) server_negotiation.cc:299] Negotiation successful
0811 09:08:04.413014 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
W20260811 09:08:04.414083 2736 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:47252
I20260811 09:08:04.415767 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 2853
Aug 11 09:08:04 dist-test-slave-xt4k krb5kdc[2650](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439284, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.28.154.129@KRBTEST.COM
I20260811 09:08:04.432299 3648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:04.416756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36366 (local address 127.28.154.129:37755)
0811 09:08:04.417373 (+ 617us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:04.417381 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:04.417405 (+ 24us) server_negotiation.cc:407] Connection header received
0811 09:08:04.417463 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:04.417469 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:04.417795 (+ 326us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:04.417995 (+ 200us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:04.419155 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:04.420205 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:04.421521 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:04.422218 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:04.422326 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:04.426506 (+ 4180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:04.426519 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:04.426534 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:08:04.426601 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:04.429488 (+ 2887us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:04.430265 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:04.430273 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:04.430275 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:04.430420 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:04.430847 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:04.430859 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:04.430863 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:04.431266 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:04.431552 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:04.431782 (+ 230us) server_negotiation.cc:299] Negotiation successful
0811 09:08:04.432003 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":453,"thread_start_us":281,"threads_started":1}
W20260811 09:08:04.433866 3092 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:36366
W20260811 09:08:04.435360 29290 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 2853: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 09:08:04.435565 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 2853
I20260811 09:08:04.464593 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 3112
Aug 11 09:08:04 dist-test-slave-xt4k krb5kdc[2650](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439284, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.28.154.130@KRBTEST.COM
I20260811 09:08:04.481972 3649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:04.465492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41210 (local address 127.28.154.130:39153)
0811 09:08:04.466230 (+ 738us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:04.466259 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:04.466297 (+ 38us) server_negotiation.cc:407] Connection header received
0811 09:08:04.466361 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:04.466368 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:04.466651 (+ 283us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:04.466793 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:04.468209 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:04.469379 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:04.470738 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:04.471326 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:04.471468 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:04.475812 (+ 4344us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:04.475826 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:04.475843 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:08:04.475887 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:04.478965 (+ 3078us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:04.479679 (+ 714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:04.479688 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:04.479690 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:04.479824 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:04.480265 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:04.480270 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:04.480272 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:04.480730 (+ 458us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:04.480978 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:04.481402 (+ 424us) server_negotiation.cc:299] Negotiation successful
0811 09:08:04.481649 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":577,"thread_start_us":465,"threads_started":1}
W20260811 09:08:04.483322 3357 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:41210
W20260811 09:08:04.484724 29290 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 3112: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 09:08:04.484889 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 3112
I20260811 09:08:04.508656 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 3378
Aug 11 09:08:04 dist-test-slave-xt4k krb5kdc[2650](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439284, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.28.154.131@KRBTEST.COM
I20260811 09:08:04.525185 3650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:04.509586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50990 (local address 127.28.154.131:42577)
0811 09:08:04.510219 (+ 633us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:04.510227 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:04.510257 (+ 30us) server_negotiation.cc:407] Connection header received
0811 09:08:04.510323 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:04.510329 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:04.510544 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:04.510670 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:04.512013 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:04.513196 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:04.514452 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:04.515036 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:04.515143 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:04.519516 (+ 4373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:04.519528 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:04.519545 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:08:04.519581 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:04.522389 (+ 2808us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:04.523065 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:04.523072 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:04.523075 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:04.523205 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:04.523605 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:04.523610 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:04.523612 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:04.524039 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:04.524279 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:04.524615 (+ 336us) server_negotiation.cc:299] Negotiation successful
0811 09:08:04.524819 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":478,"thread_start_us":303,"threads_started":1}
W20260811 09:08:04.526602 3614 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:50990
W20260811 09:08:04.527714 29290 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 3378: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 09:08:04.527843 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 3378
I20260811 09:08:04.550597 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 2673
I20260811 09:08:04.551941 2793 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:08:04.657550 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 2673
2026-08-11T09:08:04Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (3917 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439250192320-29290-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:08:04 dist-test-slave-xt4k krb5kdc[3659](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:08:04 dist-test-slave-xt4k krb5kdc[3659](info): set up 2 sockets
Aug 11 09:08:04 dist-test-slave-xt4k krb5kdc[3659](info): commencing operation
krb5kdc: starting...
W20260811 09:08:05.947503 29290 mini_kdc.cc:121] Time spent starting KDC: real 1.243s user 0.006s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439250192320-29290-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439250192320-29290-0/krb5kdc/test-user.keytab.
Aug 11 09:08:06 dist-test-slave-xt4k krb5kdc[3659](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439286, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:08:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:08:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.154.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.154.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.154.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab.
Entry for principal kudu/127.28.154.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab.
I20260811 09:08:06.180887 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439250192320-29290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439250192320-29290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439250192320-29290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.154.190:41281
--webserver_interface=127.28.154.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab
--principal=kudu/127.28.154.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:39693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.154.190:41281
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:08:06.517410 3678 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:08:06.517735 3678 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:08:06.517855 3678 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:08:06.525450 3678 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:08:06.525585 3678 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:08:06.525651 3678 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:08:06.525866 3678 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:08:06.525956 3678 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:08:06.536222 3678 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:39693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439250192320-29290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439250192320-29290-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.154.190:41281
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab
--principal=kudu/127.28.154.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.154.190:41281
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.154.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439250192320-29290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:08:06.537658 3678 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:08:06.539000 3678 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:08:06.551028 3689 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:08:06.550989 3695 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:08:06.550956 3690 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:08:06.551896 3678 server_base.cc:1034] running on GCE node
I20260811 09:08:06.552774 3678 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:08:06.554220 3678 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:08:06.555476 3678 hybrid_clock.cc:648] HybridClock initialized: now 1786439286555395 us; error 32 us; skew 500 ppm
I20260811 09:08:06.555801 3678 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:08:06 dist-test-slave-xt4k krb5kdc[3659](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439286, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:08:06.558930 3678 init.cc:377] Logged in from keytab as kudu/127.28.154.190@KRBTEST.COM (short username kudu)
I20260811 09:08:06.560501 3678 webserver.cc:460] Webserver started at http://127.28.154.190:40989/ using document root <none> and password file <none>
I20260811 09:08:06.561177 3678 fs_manager.cc:359] Metadata directory not provided
I20260811 09:08:06.561283 3678 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:08:06.561612 3678 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:08:06.564749 3678 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439250192320-29290-0/minicluster-data/master-0/data/instance:
uuid: "749949a65d1648abb7e08e3e118b55a9"
format_stamp: "Formatted at 2026-08-11 09:08:06 on dist-test-slave-xt4k"
server_key: "5fb915594bea5f049a29d0338e4f49c6"
server_key_iv: "86a78d7dab8f9db1d7dbb79d34465706"
server_key_version: "encryptionkey@0"
I20260811 09:08:06.565558 3678 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439250192320-29290-0/minicluster-data/master-0/wal/instance:
uuid: "749949a65d1648abb7e08e3e118b55a9"
format_stamp: "Formatted at 2026-08-11 09:08:06 on dist-test-slave-xt4k"
server_key: "5fb915594bea5f049a29d0338e4f49c6"
server_key_iv: "86a78d7dab8f9db1d7dbb79d34465706"
server_key_version: "encryptionkey@0"
I20260811 09:08:06.570842 3678 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 09:08:06.574414 3709 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:08:06.575732 3678 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 09:08:06.575891 3678 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439250192320-29290-0/minicluster-data/master-0/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439250192320-29290-0/minicluster-data/master-0/wal
uuid: "749949a65d1648abb7e08e3e118b55a9"
format_stamp: "Formatted at 2026-08-11 09:08:06 on dist-test-slave-xt4k"
server_key: "5fb915594bea5f049a29d0338e4f49c6"
server_key_iv: "86a78d7dab8f9db1d7dbb79d34465706"
server_key_version: "encryptionkey@0"
I20260811 09:08:06.576086 3678 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439250192320-29290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439250192320-29290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439250192320-29290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:08:06.589385 3678 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:08:06.593292 3678 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:08:06.593564 3678 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:08:06.611436 3678 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.190:41281
I20260811 09:08:06.611459 3805 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.190:41281 every 8 connection(s)
I20260811 09:08:06.612926 3678 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
I20260811 09:08:06.616820 3806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:08:06.623335 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 3678
I20260811 09:08:06.623585 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439250192320-29290-0/minicluster-data/master-0/wal/instance
I20260811 09:08:06.624437 29290 external_mini_cluster.cc:1428] Setting key 75933f7361c0752eb003fa19a46563ec
I20260811 09:08:06.628206 3806 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 749949a65d1648abb7e08e3e118b55a9: Bootstrap starting.
I20260811 09:08:06.632470 3806 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 749949a65d1648abb7e08e3e118b55a9: Neither blocks nor log segments found. Creating new log.
I20260811 09:08:06.633574 3806 log.cc:824] T 00000000000000000000000000000000 P 749949a65d1648abb7e08e3e118b55a9: Log is configured to *not* fsync() on all Append() calls
I20260811 09:08:06.636772 3806 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 749949a65d1648abb7e08e3e118b55a9: No bootstrap required, opened a new log
Aug 11 09:08:06 dist-test-slave-xt4k krb5kdc[3659](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439286, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.154.190@KRBTEST.COM
I20260811 09:08:06.646343 3806 raft_consensus.cc:348] T 00000000000000000000000000000000 P 749949a65d1648abb7e08e3e118b55a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "749949a65d1648abb7e08e3e118b55a9" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 41281 } }
I20260811 09:08:06.646741 3806 raft_consensus.cc:374] T 00000000000000000000000000000000 P 749949a65d1648abb7e08e3e118b55a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:08:06.646857 3806 raft_consensus.cc:729] T 00000000000000000000000000000000 P 749949a65d1648abb7e08e3e118b55a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 749949a65d1648abb7e08e3e118b55a9, State: Initialized, Role: FOLLOWER
I20260811 09:08:06.647540 3806 consensus_queue.cc:260] T 00000000000000000000000000000000 P 749949a65d1648abb7e08e3e118b55a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "749949a65d1648abb7e08e3e118b55a9" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 41281 } }
I20260811 09:08:06.647835 3806 raft_consensus.cc:388] T 00000000000000000000000000000000 P 749949a65d1648abb7e08e3e118b55a9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:08:06.647987 3806 raft_consensus.cc:482] T 00000000000000000000000000000000 P 749949a65d1648abb7e08e3e118b55a9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:08:06.648183 3806 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 749949a65d1648abb7e08e3e118b55a9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:08:06.650882 3810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:06.626857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46290 (local address 127.28.154.190:41281)
0811 09:08:06.627860 (+ 1003us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:06.627876 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:06.628397 (+ 521us) server_negotiation.cc:407] Connection header received
0811 09:08:06.630985 (+ 2588us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:06.631018 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:06.631558 (+ 540us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:06.632006 (+ 448us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:06.633470 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:06.635194 (+ 1724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:06.636618 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:06.637663 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:06.637852 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:06.641879 (+ 4027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:06.641890 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:06.641916 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:08:06.641966 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:06.645988 (+ 4022us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:06.646743 (+ 755us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:06.646752 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:06.646760 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:06.646979 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:06.647401 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:06.647406 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:06.647410 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:06.648212 (+ 802us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:06.648537 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:06.648981 (+ 444us) server_negotiation.cc:299] Negotiation successful
0811 09:08:06.649460 (+ 479us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":736,"thread_start_us":417,"threads_started":1}
I20260811 09:08:06.651484 3806 raft_consensus.cc:504] T 00000000000000000000000000000000 P 749949a65d1648abb7e08e3e118b55a9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "749949a65d1648abb7e08e3e118b55a9" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 41281 } }
I20260811 09:08:06.651978 3806 leader_election.cc:302] T 00000000000000000000000000000000 P 749949a65d1648abb7e08e3e118b55a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 749949a65d1648abb7e08e3e118b55a9; no voters:
I20260811 09:08:06.652611 3806 leader_election.cc:288] T 00000000000000000000000000000000 P 749949a65d1648abb7e08e3e118b55a9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:08:06.652776 3813 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 749949a65d1648abb7e08e3e118b55a9 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:08:06.653450 3813 raft_consensus.cc:686] T 00000000000000000000000000000000 P 749949a65d1648abb7e08e3e118b55a9 [term 1 LEADER]: Becoming Leader. State: Replica: 749949a65d1648abb7e08e3e118b55a9, State: Running, Role: LEADER
I20260811 09:08:06.654084 3813 consensus_queue.cc:237] T 00000000000000000000000000000000 P 749949a65d1648abb7e08e3e118b55a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "749949a65d1648abb7e08e3e118b55a9" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 41281 } }
I20260811 09:08:06.656038 3806 sys_catalog.cc:564] T 00000000000000000000000000000000 P 749949a65d1648abb7e08e3e118b55a9 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:08:06.657671 3815 sys_catalog.cc:455] T 00000000000000000000000000000000 P 749949a65d1648abb7e08e3e118b55a9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 749949a65d1648abb7e08e3e118b55a9. Latest consensus state: current_term: 1 leader_uuid: "749949a65d1648abb7e08e3e118b55a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "749949a65d1648abb7e08e3e118b55a9" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 41281 } } }
I20260811 09:08:06.658049 3815 sys_catalog.cc:458] T 00000000000000000000000000000000 P 749949a65d1648abb7e08e3e118b55a9 [sys.catalog]: This master's current role is: LEADER
I20260811 09:08:06.658556 3817 sys_catalog.cc:455] T 00000000000000000000000000000000 P 749949a65d1648abb7e08e3e118b55a9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "749949a65d1648abb7e08e3e118b55a9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "749949a65d1648abb7e08e3e118b55a9" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 41281 } } }
I20260811 09:08:06.659359 3817 sys_catalog.cc:458] T 00000000000000000000000000000000 P 749949a65d1648abb7e08e3e118b55a9 [sys.catalog]: This master's current role is: LEADER
I20260811 09:08:06.660146 3831 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:08:06.665632 3831 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:08:06.675930 3831 catalog_manager.cc:1347] Generated new cluster ID: c00e91d092684689994f75174e7b786a
I20260811 09:08:06.676071 3831 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:08:06.693675 3831 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:08:06.695124 3831 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:08:06.707764 3831 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 749949a65d1648abb7e08e3e118b55a9: Generated new TSK 0
I20260811 09:08:06.708885 3831 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.28.154.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.154.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.154.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.129.keytab.
Entry for principal kudu/127.28.154.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.129.keytab.
I20260811 09:08:06.779986 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439250192320-29290-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439250192320-29290-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439250192320-29290-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439250192320-29290-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.154.129:0
--local_ip_for_outbound_sockets=127.28.154.129
--webserver_interface=127.28.154.129
--webserver_port=0
--tserver_master_addrs=127.28.154.190:41281
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.129.keytab
--principal=kudu/127.28.154.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:39693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:08:07.115643 3854 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:08:07.115967 3854 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:08:07.116068 3854 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:08:07.123757 3854 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:08:07.123885 3854 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:08:07.124027 3854 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.154.129
I20260811 09:08:07.134006 3854 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:39693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439250192320-29290-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439250192320-29290-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.129.keytab
--principal=kudu/127.28.154.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.154.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439250192320-29290-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.154.129
--webserver_port=0
--tserver_master_addrs=127.28.154.190:41281
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.154.129
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439250192320-29290-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:08:07.135303 3854 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:08:07.136664 3854 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:08:07.149634 3864 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:08:07.149664 3867 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:08:07.150164 3877 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:08:07.150681 3854 server_base.cc:1034] running on GCE node
I20260811 09:08:07.151372 3854 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:08:07.152375 3854 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:08:07.153597 3854 hybrid_clock.cc:648] HybridClock initialized: now 1786439287153549 us; error 42 us; skew 500 ppm
I20260811 09:08:07.153928 3854 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:08:07 dist-test-slave-xt4k krb5kdc[3659](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439287, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:08:07.157253 3854 init.cc:377] Logged in from keytab as kudu/127.28.154.129@KRBTEST.COM (short username kudu)
I20260811 09:08:07.158952 3854 webserver.cc:460] Webserver started at http://127.28.154.129:42481/ using document root <none> and password file <none>
I20260811 09:08:07.159817 3854 fs_manager.cc:359] Metadata directory not provided
I20260811 09:08:07.159963 3854 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:08:07.160305 3854 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:08:07.163532 3854 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439250192320-29290-0/minicluster-data/ts-0/data/instance:
uuid: "cb54909dd6894111a6f0d8fa918c720c"
format_stamp: "Formatted at 2026-08-11 09:08:07 on dist-test-slave-xt4k"
server_key: "c7895483fa7e22097f9a16f959eb305d"
server_key_iv: "ed571719eec7f5c8e6f6ae1e8d108c30"
server_key_version: "encryptionkey@0"
I20260811 09:08:07.164381 3854 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439250192320-29290-0/minicluster-data/ts-0/wal/instance:
uuid: "cb54909dd6894111a6f0d8fa918c720c"
format_stamp: "Formatted at 2026-08-11 09:08:07 on dist-test-slave-xt4k"
server_key: "c7895483fa7e22097f9a16f959eb305d"
server_key_iv: "ed571719eec7f5c8e6f6ae1e8d108c30"
server_key_version: "encryptionkey@0"
I20260811 09:08:07.169939 3854 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260811 09:08:07.173862 3890 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:08:07.175360 3854 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 09:08:07.175552 3854 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439250192320-29290-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439250192320-29290-0/minicluster-data/ts-0/wal
uuid: "cb54909dd6894111a6f0d8fa918c720c"
format_stamp: "Formatted at 2026-08-11 09:08:07 on dist-test-slave-xt4k"
server_key: "c7895483fa7e22097f9a16f959eb305d"
server_key_iv: "ed571719eec7f5c8e6f6ae1e8d108c30"
server_key_version: "encryptionkey@0"
I20260811 09:08:07.175766 3854 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439250192320-29290-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439250192320-29290-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439250192320-29290-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:08:07.206305 3854 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:08:07.211246 3854 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:08:07.211542 3854 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:08:07.212608 3854 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:08:07.214027 3854 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:08:07.214108 3854 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:08:07.214231 3854 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:08:07.214296 3854 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:08:07.248262 3854 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.129:40023
I20260811 09:08:07.248317 4104 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.129:40023 every 8 connection(s)
I20260811 09:08:07.249657 3854 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439250192320-29290-0/minicluster-data/ts-0/data/info.pb
I20260811 09:08:07.253819 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 3854
I20260811 09:08:07.254026 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439250192320-29290-0/minicluster-data/ts-0/wal/instance
I20260811 09:08:07.254853 29290 external_mini_cluster.cc:1428] Setting key eda37ea9d054082355b03cd373c11a77
Aug 11 09:08:07 dist-test-slave-xt4k krb5kdc[3659](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439287, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.129@KRBTEST.COM for kudu/127.28.154.190@KRBTEST.COM
I20260811 09:08:07.273034 4111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:07.252222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.129:59927 (local address 127.28.154.190:41281)
0811 09:08:07.252708 (+ 486us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:07.252713 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:07.253776 (+ 1063us) server_negotiation.cc:407] Connection header received
0811 09:08:07.256161 (+ 2385us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:07.256167 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:07.256244 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:07.256356 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:07.258306 (+ 1950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:07.259037 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:07.261188 (+ 2151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:07.261668 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:07.261784 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:07.265817 (+ 4033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:07.265825 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:07.265830 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:08:07.265872 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:07.269122 (+ 3250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:07.270081 (+ 959us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:07.270087 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:07.270091 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:07.270253 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:07.270705 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:07.270710 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:07.270712 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:07.271206 (+ 494us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:07.271374 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:07.272430 (+ 1056us) server_negotiation.cc:299] Negotiation successful
0811 09:08:07.272693 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":373,"thread_start_us":261,"threads_started":1}
I20260811 09:08:07.273988 4112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:07.252777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:41281 (local address 127.28.154.129:59927)
0811 09:08:07.253567 (+ 790us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:07.253631 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:07.255838 (+ 2207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:07.256638 (+ 800us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:07.256649 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:07.257306 (+ 657us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:08:07.258123 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:07.258150 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:07.259205 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:07.259210 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:07.261036 (+ 1826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:07.261051 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:07.261905 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:07.261912 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:07.262202 (+ 290us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:07.262935 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:08:07.262963 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:08:07.265530 (+ 2567us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:08:07.269296 (+ 3766us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:08:07.269303 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:08:07.269317 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:08:07.269933 (+ 616us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:08:07.270363 (+ 430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:08:07.270366 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:08:07.270368 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:08:07.270578 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:08:07.271412 (+ 834us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:08:07.271420 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:08:07.271939 (+ 519us) client_negotiation.cc:769] Sending connection context
0811 09:08:07.272214 (+ 275us) client_negotiation.cc:240] Negotiation successful
0811 09:08:07.272777 (+ 563us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":278,"threads_started":1}
I20260811 09:08:07.275707 4105 heartbeater.cc:347] Connected to a master server at 127.28.154.190:41281
I20260811 09:08:07.276081 4105 heartbeater.cc:464] Registering TS with master...
I20260811 09:08:07.277086 4105 heartbeater.cc:511] Master 127.28.154.190:41281 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.28.154.130@KRBTEST.COM; defaulting to no policy
I20260811 09:08:07.279919 3746 ts_manager.cc:194] Registered new tserver with Master: cb54909dd6894111a6f0d8fa918c720c (127.28.154.129:40023)
I20260811 09:08:07.281523 3746 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.154.129@KRBTEST.COM'} at 127.28.154.129:59927
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.154.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.154.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.130.keytab.
Entry for principal kudu/127.28.154.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.130.keytab.
I20260811 09:08:07.309363 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439250192320-29290-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439250192320-29290-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439250192320-29290-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439250192320-29290-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.154.130:0
--local_ip_for_outbound_sockets=127.28.154.130
--webserver_interface=127.28.154.130
--webserver_port=0
--tserver_master_addrs=127.28.154.190:41281
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.130.keytab
--principal=kudu/127.28.154.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:39693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:08:07.636380 4117 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:08:07.636687 4117 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:08:07.636811 4117 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:08:07.644017 4117 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:08:07.644131 4117 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:08:07.644277 4117 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.154.130
I20260811 09:08:07.654032 4117 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:39693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439250192320-29290-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439250192320-29290-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.130.keytab
--principal=kudu/127.28.154.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.154.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439250192320-29290-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.154.130
--webserver_port=0
--tserver_master_addrs=127.28.154.190:41281
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.154.130
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439250192320-29290-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:08:07.655297 4117 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:08:07.656687 4117 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:08:07.670397 4124 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:08:07.670589 4125 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:08:07.670938 4127 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:08:07.671203 4117 server_base.cc:1034] running on GCE node
I20260811 09:08:07.671854 4117 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:08:07.672921 4117 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:08:07.674214 4117 hybrid_clock.cc:648] HybridClock initialized: now 1786439287674117 us; error 87 us; skew 500 ppm
I20260811 09:08:07.674526 4117 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:08:07 dist-test-slave-xt4k krb5kdc[3659](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439287, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:08:07.678059 4117 init.cc:377] Logged in from keytab as kudu/127.28.154.130@KRBTEST.COM (short username kudu)
I20260811 09:08:07.679749 4117 webserver.cc:460] Webserver started at http://127.28.154.130:45611/ using document root <none> and password file <none>
I20260811 09:08:07.680585 4117 fs_manager.cc:359] Metadata directory not provided
I20260811 09:08:07.680697 4117 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:08:07.681041 4117 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:08:07.684020 4117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439250192320-29290-0/minicluster-data/ts-1/data/instance:
uuid: "e92cf1fde93a4456970c2ee943066f60"
format_stamp: "Formatted at 2026-08-11 09:08:07 on dist-test-slave-xt4k"
server_key: "95c69783344c7b5129e0ffdcb7a27c99"
server_key_iv: "f2f66c6ad7c9b3cb9053c3730b25ab7a"
server_key_version: "encryptionkey@0"
I20260811 09:08:07.684810 4117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439250192320-29290-0/minicluster-data/ts-1/wal/instance:
uuid: "e92cf1fde93a4456970c2ee943066f60"
format_stamp: "Formatted at 2026-08-11 09:08:07 on dist-test-slave-xt4k"
server_key: "95c69783344c7b5129e0ffdcb7a27c99"
server_key_iv: "f2f66c6ad7c9b3cb9053c3730b25ab7a"
server_key_version: "encryptionkey@0"
I20260811 09:08:07.690117 4117 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 09:08:07.694097 4143 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:08:07.695494 4117 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 09:08:07.695720 4117 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439250192320-29290-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439250192320-29290-0/minicluster-data/ts-1/wal
uuid: "e92cf1fde93a4456970c2ee943066f60"
format_stamp: "Formatted at 2026-08-11 09:08:07 on dist-test-slave-xt4k"
server_key: "95c69783344c7b5129e0ffdcb7a27c99"
server_key_iv: "f2f66c6ad7c9b3cb9053c3730b25ab7a"
server_key_version: "encryptionkey@0"
I20260811 09:08:07.695919 4117 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439250192320-29290-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439250192320-29290-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439250192320-29290-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:08:07.707890 4117 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:08:07.712177 4117 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:08:07.712494 4117 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:08:07.713474 4117 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:08:07.714874 4117 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:08:07.714953 4117 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:08:07.715178 4117 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:08:07.715261 4117 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:08:07.747637 4117 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.130:35963
I20260811 09:08:07.747654 4336 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.130:35963 every 8 connection(s)
I20260811 09:08:07.748960 4117 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439250192320-29290-0/minicluster-data/ts-1/data/info.pb
I20260811 09:08:07.751157 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 4117
I20260811 09:08:07.751322 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439250192320-29290-0/minicluster-data/ts-1/wal/instance
I20260811 09:08:07.751858 29290 external_mini_cluster.cc:1428] Setting key bfecbda91e66517b03cad5f69d8856b3
Aug 11 09:08:07 dist-test-slave-xt4k krb5kdc[3659](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439287, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.130@KRBTEST.COM for kudu/127.28.154.190@KRBTEST.COM
I20260811 09:08:07.771497 4111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:07.751626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.130:37411 (local address 127.28.154.190:41281)
0811 09:08:07.751844 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:07.751851 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:07.753076 (+ 1225us) server_negotiation.cc:407] Connection header received
0811 09:08:07.755418 (+ 2342us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:07.755422 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:07.755521 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:07.755602 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:07.757638 (+ 2036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:07.758379 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:07.760517 (+ 2138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:07.760980 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:07.761115 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:07.765205 (+ 4090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:07.765210 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:07.765214 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:08:07.765249 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:07.767876 (+ 2627us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:07.768941 (+ 1065us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:07.768946 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:07.768948 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:07.769110 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:07.769565 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:07.769569 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:07.769571 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:07.769877 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:07.770017 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:07.770889 (+ 872us) server_negotiation.cc:299] Negotiation successful
0811 09:08:07.771149 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 09:08:07.772616 4339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:07.752023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:41281 (local address 127.28.154.130:37411)
0811 09:08:07.752839 (+ 816us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:07.752950 (+ 111us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:07.755131 (+ 2181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:07.755896 (+ 765us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:07.755907 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:07.756667 (+ 760us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:08:07.757463 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:07.757488 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:07.758539 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:07.758544 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:07.760316 (+ 1772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:07.760361 (+ 45us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:07.761200 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:07.761207 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:07.761481 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:07.762196 (+ 715us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:08:07.762225 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:08:07.764908 (+ 2683us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:08:07.768121 (+ 3213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:08:07.768150 (+ 29us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:08:07.768167 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:08:07.768792 (+ 625us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:08:07.769228 (+ 436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:08:07.769233 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:08:07.769238 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:08:07.769442 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:08:07.770005 (+ 563us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:08:07.770014 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:08:07.770464 (+ 450us) client_negotiation.cc:769] Sending connection context
0811 09:08:07.770745 (+ 281us) client_negotiation.cc:240] Negotiation successful
0811 09:08:07.771314 (+ 569us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":510,"thread_start_us":319,"threads_started":1}
WARNING: no policy specified for kudu/127.28.154.131@KRBTEST.COM; defaulting to no policy
I20260811 09:08:07.774307 4337 heartbeater.cc:347] Connected to a master server at 127.28.154.190:41281
I20260811 09:08:07.774623 4337 heartbeater.cc:464] Registering TS with master...
I20260811 09:08:07.775367 4337 heartbeater.cc:511] Master 127.28.154.190:41281 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.154.131@KRBTEST.COM" created.
I20260811 09:08:07.777014 3746 ts_manager.cc:194] Registered new tserver with Master: e92cf1fde93a4456970c2ee943066f60 (127.28.154.130:35963)
I20260811 09:08:07.777978 3746 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.154.130@KRBTEST.COM'} at 127.28.154.130:37411
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.154.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.131.keytab.
Entry for principal kudu/127.28.154.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.131.keytab.
I20260811 09:08:07.803689 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439250192320-29290-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439250192320-29290-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439250192320-29290-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439250192320-29290-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.154.131:0
--local_ip_for_outbound_sockets=127.28.154.131
--webserver_interface=127.28.154.131
--webserver_port=0
--tserver_master_addrs=127.28.154.190:41281
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.131.keytab
--principal=kudu/127.28.154.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:39693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:08:08.142242 4343 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:08:08.142570 4343 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:08:08.142694 4343 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:08:08.150314 4343 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:08:08.150439 4343 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:08:08.150586 4343 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.154.131
I20260811 09:08:08.160784 4343 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:39693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439250192320-29290-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439250192320-29290-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.131.keytab
--principal=kudu/127.28.154.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.154.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439250192320-29290-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.154.131
--webserver_port=0
--tserver_master_addrs=127.28.154.190:41281
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.154.131
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439250192320-29290-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:08:08.162221 4343 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:08:08.163617 4343 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:08:08.178164 4356 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:08:08.178220 4349 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:08:08.178252 4352 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:08:08.179008 4343 server_base.cc:1034] running on GCE node
I20260811 09:08:08.180226 4343 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:08:08.181526 4343 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:08:08.182790 4343 hybrid_clock.cc:648] HybridClock initialized: now 1786439288182723 us; error 65 us; skew 500 ppm
I20260811 09:08:08.183143 4343 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:08:08 dist-test-slave-xt4k krb5kdc[3659](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439288, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:08:08.186704 4343 init.cc:377] Logged in from keytab as kudu/127.28.154.131@KRBTEST.COM (short username kudu)
I20260811 09:08:08.188373 4343 webserver.cc:460] Webserver started at http://127.28.154.131:41137/ using document root <none> and password file <none>
I20260811 09:08:08.189109 4343 fs_manager.cc:359] Metadata directory not provided
I20260811 09:08:08.189231 4343 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:08:08.189556 4343 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:08:08.192662 4343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439250192320-29290-0/minicluster-data/ts-2/data/instance:
uuid: "4bf0fd1da1564d9f9b2e147f2811b0ae"
format_stamp: "Formatted at 2026-08-11 09:08:08 on dist-test-slave-xt4k"
server_key: "23f48b3510566625d1e917f8131c6ca2"
server_key_iv: "b655a9d14f513e081ce28b4fd7e5b2e2"
server_key_version: "encryptionkey@0"
I20260811 09:08:08.193490 4343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439250192320-29290-0/minicluster-data/ts-2/wal/instance:
uuid: "4bf0fd1da1564d9f9b2e147f2811b0ae"
format_stamp: "Formatted at 2026-08-11 09:08:08 on dist-test-slave-xt4k"
server_key: "23f48b3510566625d1e917f8131c6ca2"
server_key_iv: "b655a9d14f513e081ce28b4fd7e5b2e2"
server_key_version: "encryptionkey@0"
I20260811 09:08:08.199164 4343 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 09:08:08.203200 4372 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:08:08.204865 4343 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 09:08:08.205062 4343 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439250192320-29290-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439250192320-29290-0/minicluster-data/ts-2/wal
uuid: "4bf0fd1da1564d9f9b2e147f2811b0ae"
format_stamp: "Formatted at 2026-08-11 09:08:08 on dist-test-slave-xt4k"
server_key: "23f48b3510566625d1e917f8131c6ca2"
server_key_iv: "b655a9d14f513e081ce28b4fd7e5b2e2"
server_key_version: "encryptionkey@0"
I20260811 09:08:08.205281 4343 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439250192320-29290-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439250192320-29290-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439250192320-29290-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:08:08.220937 4343 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:08:08.225488 4343 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:08:08.225795 4343 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:08:08.226821 4343 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:08:08.228241 4343 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:08:08.228324 4343 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:08:08.228487 4343 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:08:08.228554 4343 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:08:08.263443 4343 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.131:46397
I20260811 09:08:08.263495 4581 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.131:46397 every 8 connection(s)
I20260811 09:08:08.264782 4343 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439250192320-29290-0/minicluster-data/ts-2/data/info.pb
I20260811 09:08:08.267917 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 4343
I20260811 09:08:08.268074 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786439250192320-29290-0/minicluster-data/ts-2/wal/instance
I20260811 09:08:08.268559 29290 external_mini_cluster.cc:1428] Setting key 09dea11f3a7c4c0ffbc33dd239364688
Aug 11 09:08:08 dist-test-slave-xt4k krb5kdc[3659](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439288, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.131@KRBTEST.COM for kudu/127.28.154.190@KRBTEST.COM
I20260811 09:08:08.284799 4105 heartbeater.cc:503] Master 127.28.154.190:41281 was elected leader, sending a full tablet report...
I20260811 09:08:08.287523 4111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:08.267393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.131:55435 (local address 127.28.154.190:41281)
0811 09:08:08.267633 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:08.267641 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:08.268864 (+ 1223us) server_negotiation.cc:407] Connection header received
0811 09:08:08.270670 (+ 1806us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:08.270675 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:08.270758 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:08.270861 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:08.273122 (+ 2261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:08.273958 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:08.275718 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:08.276154 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:08.276270 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:08.280099 (+ 3829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:08.280105 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:08.280108 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:08:08.280144 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:08.283810 (+ 3666us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:08.284999 (+ 1189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:08.285004 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:08.285007 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:08.285193 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:08.285693 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:08.285698 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:08.285702 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:08.286073 (+ 371us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:08.286237 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:08.287083 (+ 846us) server_negotiation.cc:299] Negotiation successful
0811 09:08:08.287260 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 09:08:08.288537 4588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:08.267849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:41281 (local address 127.28.154.131:55435)
0811 09:08:08.268688 (+ 839us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:08.268733 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:08.270409 (+ 1676us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:08.271065 (+ 656us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:08.271074 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:08.271621 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:08:08.272781 (+ 1160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:08.272814 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:08.274153 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:08.274160 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:08.275515 (+ 1355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:08.275569 (+ 54us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:08.276450 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:08.276457 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:08.276745 (+ 288us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:08.277399 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:08:08.277423 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:08:08.279850 (+ 2427us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:08:08.283996 (+ 4146us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:08:08.284005 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:08:08.284035 (+ 30us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:08:08.284866 (+ 831us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:08:08.285335 (+ 469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:08:08.285341 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:08:08.285345 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:08:08.285568 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:08:08.286217 (+ 649us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:08:08.286227 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:08:08.286713 (+ 486us) client_negotiation.cc:769] Sending connection context
0811 09:08:08.286951 (+ 238us) client_negotiation.cc:240] Negotiation successful
0811 09:08:08.287394 (+ 443us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":547,"thread_start_us":321,"threads_started":1}
I20260811 09:08:08.290045 4582 heartbeater.cc:347] Connected to a master server at 127.28.154.190:41281
I20260811 09:08:08.290357 4582 heartbeater.cc:464] Registering TS with master...
I20260811 09:08:08.291285 4582 heartbeater.cc:511] Master 127.28.154.190:41281 requested a full tablet report, sending...
I20260811 09:08:08.293078 3746 ts_manager.cc:194] Registered new tserver with Master: 4bf0fd1da1564d9f9b2e147f2811b0ae (127.28.154.131:46397)
I20260811 09:08:08.293938 3746 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.154.131@KRBTEST.COM'} at 127.28.154.131:55435
I20260811 09:08:08.297906 29290 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 09:08:08 dist-test-slave-xt4k krb5kdc[3659](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439288, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 09:08:08 dist-test-slave-xt4k krb5kdc[3659](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439288, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.154.129@KRBTEST.COM
I20260811 09:08:08.352298 4601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:08.336909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46276 (local address 127.28.154.129:40023)
0811 09:08:08.337657 (+ 748us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:08.337664 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:08.337693 (+ 29us) server_negotiation.cc:407] Connection header received
0811 09:08:08.337755 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:08.337762 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:08.338000 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:08.338146 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:08.339622 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:08.340738 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:08.341947 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:08.342546 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:08.342662 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:08.346658 (+ 3996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:08.346668 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:08.346684 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:08:08.346721 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:08.349751 (+ 3030us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:08.350403 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:08.350411 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:08.350413 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:08.350568 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:08.350904 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:08.350908 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:08.350910 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:08.351268 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:08.351524 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:08.351805 (+ 281us) server_negotiation.cc:299] Negotiation successful
0811 09:08:08.352005 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":587,"thread_start_us":369,"threads_started":1}
Aug 11 09:08:08 dist-test-slave-xt4k krb5kdc[3659](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439288, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.154.190@KRBTEST.COM
I20260811 09:08:08.370638 4111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:08.357050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46306 (local address 127.28.154.190:41281)
0811 09:08:08.357240 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:08.357246 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:08.357458 (+ 212us) server_negotiation.cc:407] Connection header received
0811 09:08:08.357620 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:08.357625 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:08.357691 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:08.357774 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:08.359009 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:08.359743 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:08.361092 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:08.361564 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:08.361677 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:08.365420 (+ 3743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:08.365425 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:08.365429 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:08:08.365468 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:08.367856 (+ 2388us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:08.368678 (+ 822us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:08.368684 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:08.368687 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:08.368833 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:08.369187 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:08.369191 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:08.369193 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:08.369525 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:08.369763 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:08.370105 (+ 342us) server_negotiation.cc:299] Negotiation successful
0811 09:08:08.370317 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
W20260811 09:08:08.371330 3746 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46306
I20260811 09:08:08.372375 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 3854
I20260811 09:08:08.385886 4601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:08.373375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46284 (local address 127.28.154.129:40023)
0811 09:08:08.373557 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:08.373562 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:08.373802 (+ 240us) server_negotiation.cc:407] Connection header received
0811 09:08:08.374004 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:08.374008 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:08.374105 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:08.374187 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:08.375386 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:08.376084 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:08.377367 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:08.377846 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:08.377950 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:08.380466 (+ 2516us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:08.380472 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:08.380476 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:08:08.380538 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:08.383102 (+ 2564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:08.383813 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:08.383818 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:08.383821 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:08.383956 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:08.384388 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:08.384394 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:08.384398 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:08.384740 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:08.385005 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:08.385341 (+ 336us) server_negotiation.cc:299] Negotiation successful
0811 09:08:08.385587 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 09:08:08.386510 4096 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:08:08.469036 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 3854
I20260811 09:08:08.490969 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 4117
Aug 11 09:08:08 dist-test-slave-xt4k krb5kdc[3659](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439288, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.154.130@KRBTEST.COM
I20260811 09:08:08.507730 4611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:08.491935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36486 (local address 127.28.154.130:35963)
0811 09:08:08.492607 (+ 672us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:08.492614 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:08.492640 (+ 26us) server_negotiation.cc:407] Connection header received
0811 09:08:08.492703 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:08.492710 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:08.492922 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:08.493065 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:08.494325 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:08.495434 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:08.497062 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:08.497659 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:08.497785 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:08.502083 (+ 4298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:08.502096 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:08.502112 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:08:08.502149 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:08.505010 (+ 2861us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:08.505669 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:08.505677 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:08.505679 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:08.505802 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:08.506163 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:08.506168 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:08.506170 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:08.506522 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:08.506825 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:08.507135 (+ 310us) server_negotiation.cc:299] Negotiation successful
0811 09:08:08.507396 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":524,"thread_start_us":402,"threads_started":1}
I20260811 09:08:08.509107 4329 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:08:08.592042 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 4117
I20260811 09:08:08.624063 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 4343
Aug 11 09:08:08 dist-test-slave-xt4k krb5kdc[3659](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439288, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.154.131@KRBTEST.COM
I20260811 09:08:08.642765 4613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:08.625142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49830 (local address 127.28.154.131:46397)
0811 09:08:08.625744 (+ 602us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:08.625751 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:08.625780 (+ 29us) server_negotiation.cc:407] Connection header received
0811 09:08:08.625840 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:08.625847 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:08.626079 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:08.626208 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:08.627452 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:08.628928 (+ 1476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:08.630282 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:08.630993 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:08.631108 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:08.637112 (+ 6004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:08.637131 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:08.637149 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:08:08.637192 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:08.639946 (+ 2754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:08.640792 (+ 846us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:08.640800 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:08.640802 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:08.640929 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:08.641228 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:08.641231 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:08.641233 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:08.641607 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:08.641866 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:08.642173 (+ 307us) server_negotiation.cc:299] Negotiation successful
0811 09:08:08.642399 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":461,"thread_start_us":312,"threads_started":1}
I20260811 09:08:08.644134 4575 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:08:08.726817 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 4343
I20260811 09:08:08.747593 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 3678
I20260811 09:08:08.748960 3800 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:08:08.859236 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 3678
2026-08-11T09:08:08Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (4201 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439250192320-29290-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:08:08 dist-test-slave-xt4k krb5kdc[4618](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:08:08 dist-test-slave-xt4k krb5kdc[4618](info): set up 2 sockets
Aug 11 09:08:08 dist-test-slave-xt4k krb5kdc[4618](info): commencing operation
krb5kdc: starting...
W20260811 09:08:10.130118 29290 mini_kdc.cc:121] Time spent starting KDC: real 1.225s user 0.000s sys 0.026s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439250192320-29290-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439250192320-29290-0/krb5kdc/test-user.keytab.
Aug 11 09:08:10 dist-test-slave-xt4k krb5kdc[4618](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439290, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:08:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:08:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.154.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.154.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.154.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab.
Entry for principal kudu/127.28.154.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab.
I20260811 09:08:10.371850 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439250192320-29290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439250192320-29290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439250192320-29290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.154.190:34691
--webserver_interface=127.28.154.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab
--principal=kudu/127.28.154.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:43515
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.154.190:34691
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:08:10.727691 4642 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:08:10.728035 4642 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:08:10.728166 4642 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:08:10.737520 4642 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:08:10.737848 4642 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:08:10.738034 4642 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:08:10.738144 4642 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:08:10.738212 4642 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:08:10.748847 4642 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:43515
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439250192320-29290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439250192320-29290-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.154.190:34691
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab
--principal=kudu/127.28.154.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.154.190:34691
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.154.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439250192320-29290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:08:10.750227 4642 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:08:10.751546 4642 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:08:10.763471 4655 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:08:10.763471 4659 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:08:10.763619 4652 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:08:10.764135 4642 server_base.cc:1034] running on GCE node
I20260811 09:08:10.765055 4642 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:08:10.766577 4642 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:08:10.767815 4642 hybrid_clock.cc:648] HybridClock initialized: now 1786439290767758 us; error 47 us; skew 500 ppm
I20260811 09:08:10.768253 4642 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:08:10 dist-test-slave-xt4k krb5kdc[4618](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439290, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:08:10.772040 4642 init.cc:377] Logged in from keytab as kudu/127.28.154.190@KRBTEST.COM (short username kudu)
I20260811 09:08:10.773471 4642 fs_manager.cc:359] Metadata directory not provided
I20260811 09:08:10.773598 4642 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:08:10.774171 4642 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:08:10.777539 4642 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439250192320-29290-0/minicluster-data/master-0/data/instance:
uuid: "eeabb7f4c99e4c14813269d0e0bb1761"
format_stamp: "Formatted at 2026-08-11 09:08:10 on dist-test-slave-xt4k"
server_key: "8f4b60a75fc3fd764b5479ad83c9ae81"
server_key_iv: "674749836fd4931f02e5d9db5d501568"
server_key_version: "encryptionkey@0"
I20260811 09:08:10.778385 4642 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439250192320-29290-0/minicluster-data/master-0/wal/instance:
uuid: "eeabb7f4c99e4c14813269d0e0bb1761"
format_stamp: "Formatted at 2026-08-11 09:08:10 on dist-test-slave-xt4k"
server_key: "8f4b60a75fc3fd764b5479ad83c9ae81"
server_key_iv: "674749836fd4931f02e5d9db5d501568"
server_key_version: "encryptionkey@0"
I20260811 09:08:10.783931 4642 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 09:08:10.787988 4671 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:08:10.789537 4642 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 09:08:10.789852 4642 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439250192320-29290-0/minicluster-data/master-0/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439250192320-29290-0/minicluster-data/master-0/wal
uuid: "eeabb7f4c99e4c14813269d0e0bb1761"
format_stamp: "Formatted at 2026-08-11 09:08:10 on dist-test-slave-xt4k"
server_key: "8f4b60a75fc3fd764b5479ad83c9ae81"
server_key_iv: "674749836fd4931f02e5d9db5d501568"
server_key_version: "encryptionkey@0"
I20260811 09:08:10.790122 4642 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439250192320-29290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439250192320-29290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439250192320-29290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:08:10.810390 4642 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:08:10.814915 4642 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:08:10.815197 4642 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:08:10.832742 4642 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.190:34691
I20260811 09:08:10.832719 4762 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.190:34691 every 8 connection(s)
I20260811 09:08:10.833685 4642 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
I20260811 09:08:10.835965 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 4642
I20260811 09:08:10.836149 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439250192320-29290-0/minicluster-data/master-0/wal/instance
I20260811 09:08:10.836848 29290 external_mini_cluster.cc:1428] Setting key a5614a8d75e9d75c617e5387a9e384ab
I20260811 09:08:10.838222 4763 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 09:08:10 dist-test-slave-xt4k krb5kdc[4618](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439290, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.154.190@KRBTEST.COM
I20260811 09:08:10.852275 4763 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eeabb7f4c99e4c14813269d0e0bb1761: Bootstrap starting.
I20260811 09:08:10.856628 4763 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eeabb7f4c99e4c14813269d0e0bb1761: Neither blocks nor log segments found. Creating new log.
I20260811 09:08:10.857968 4763 log.cc:824] T 00000000000000000000000000000000 P eeabb7f4c99e4c14813269d0e0bb1761: Log is configured to *not* fsync() on all Append() calls
I20260811 09:08:10.861050 4763 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eeabb7f4c99e4c14813269d0e0bb1761: No bootstrap required, opened a new log
I20260811 09:08:10.861583 4767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:10.838824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43342 (local address 127.28.154.190:34691)
0811 09:08:10.839655 (+ 831us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:10.839670 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:10.839724 (+ 54us) server_negotiation.cc:407] Connection header received
0811 09:08:10.841724 (+ 2000us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:10.841755 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:10.842305 (+ 550us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:10.842794 (+ 489us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:10.844294 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:10.846363 (+ 2069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:10.847708 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:10.848816 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:10.849040 (+ 224us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:10.853087 (+ 4047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:10.853100 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:10.853124 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:08:10.853175 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:10.857342 (+ 4167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:10.858135 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:10.858144 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:10.858154 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:10.858364 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:10.858772 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:10.858776 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:10.858782 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:10.859372 (+ 590us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:10.859654 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:10.860138 (+ 484us) server_negotiation.cc:299] Negotiation successful
0811 09:08:10.860607 (+ 469us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":593,"thread_start_us":339,"threads_started":1}
I20260811 09:08:10.869228 4763 raft_consensus.cc:348] T 00000000000000000000000000000000 P eeabb7f4c99e4c14813269d0e0bb1761 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eeabb7f4c99e4c14813269d0e0bb1761" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 34691 } }
I20260811 09:08:10.869783 4763 raft_consensus.cc:374] T 00000000000000000000000000000000 P eeabb7f4c99e4c14813269d0e0bb1761 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:08:10.869894 4763 raft_consensus.cc:729] T 00000000000000000000000000000000 P eeabb7f4c99e4c14813269d0e0bb1761 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eeabb7f4c99e4c14813269d0e0bb1761, State: Initialized, Role: FOLLOWER
I20260811 09:08:10.870548 4763 consensus_queue.cc:260] T 00000000000000000000000000000000 P eeabb7f4c99e4c14813269d0e0bb1761 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eeabb7f4c99e4c14813269d0e0bb1761" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 34691 } }
I20260811 09:08:10.870802 4763 raft_consensus.cc:388] T 00000000000000000000000000000000 P eeabb7f4c99e4c14813269d0e0bb1761 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:08:10.870944 4763 raft_consensus.cc:482] T 00000000000000000000000000000000 P eeabb7f4c99e4c14813269d0e0bb1761 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:08:10.871145 4763 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eeabb7f4c99e4c14813269d0e0bb1761 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:08:10.873449 4763 raft_consensus.cc:504] T 00000000000000000000000000000000 P eeabb7f4c99e4c14813269d0e0bb1761 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eeabb7f4c99e4c14813269d0e0bb1761" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 34691 } }
I20260811 09:08:10.873978 4763 leader_election.cc:302] T 00000000000000000000000000000000 P eeabb7f4c99e4c14813269d0e0bb1761 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eeabb7f4c99e4c14813269d0e0bb1761; no voters:
I20260811 09:08:10.874600 4763 leader_election.cc:288] T 00000000000000000000000000000000 P eeabb7f4c99e4c14813269d0e0bb1761 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:08:10.874764 4770 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eeabb7f4c99e4c14813269d0e0bb1761 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:08:10.875411 4770 raft_consensus.cc:686] T 00000000000000000000000000000000 P eeabb7f4c99e4c14813269d0e0bb1761 [term 1 LEADER]: Becoming Leader. State: Replica: eeabb7f4c99e4c14813269d0e0bb1761, State: Running, Role: LEADER
I20260811 09:08:10.875960 4770 consensus_queue.cc:237] T 00000000000000000000000000000000 P eeabb7f4c99e4c14813269d0e0bb1761 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eeabb7f4c99e4c14813269d0e0bb1761" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 34691 } }
I20260811 09:08:10.876816 4763 sys_catalog.cc:564] T 00000000000000000000000000000000 P eeabb7f4c99e4c14813269d0e0bb1761 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:08:10.879190 4776 sys_catalog.cc:455] T 00000000000000000000000000000000 P eeabb7f4c99e4c14813269d0e0bb1761 [sys.catalog]: SysCatalogTable state changed. Reason: New leader eeabb7f4c99e4c14813269d0e0bb1761. Latest consensus state: current_term: 1 leader_uuid: "eeabb7f4c99e4c14813269d0e0bb1761" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eeabb7f4c99e4c14813269d0e0bb1761" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 34691 } } }
I20260811 09:08:10.879676 4776 sys_catalog.cc:458] T 00000000000000000000000000000000 P eeabb7f4c99e4c14813269d0e0bb1761 [sys.catalog]: This master's current role is: LEADER
I20260811 09:08:10.879433 4774 sys_catalog.cc:455] T 00000000000000000000000000000000 P eeabb7f4c99e4c14813269d0e0bb1761 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eeabb7f4c99e4c14813269d0e0bb1761" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eeabb7f4c99e4c14813269d0e0bb1761" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 34691 } } }
I20260811 09:08:10.879789 4774 sys_catalog.cc:458] T 00000000000000000000000000000000 P eeabb7f4c99e4c14813269d0e0bb1761 [sys.catalog]: This master's current role is: LEADER
I20260811 09:08:10.886123 4784 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:08:10.891821 4784 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:08:10.902148 4784 catalog_manager.cc:1347] Generated new cluster ID: 97e32b1aa14f4c5fb9d98a7214e601d6
I20260811 09:08:10.902276 4784 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:08:10.923417 4784 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:08:10.924825 4784 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:08:10.931479 4784 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eeabb7f4c99e4c14813269d0e0bb1761: Generated new TSK 0
I20260811 09:08:10.932670 4784 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.28.154.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.154.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.154.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.129.keytab.
Entry for principal kudu/127.28.154.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.129.keytab.
I20260811 09:08:10.998394 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439250192320-29290-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439250192320-29290-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439250192320-29290-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439250192320-29290-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.154.129:0
--local_ip_for_outbound_sockets=127.28.154.129
--webserver_interface=127.28.154.129
--webserver_port=0
--tserver_master_addrs=127.28.154.190:34691
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.129.keytab
--principal=kudu/127.28.154.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:43515
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:08:11.351926 4811 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:08:11.352293 4811 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:08:11.352473 4811 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:08:11.362128 4811 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:08:11.362267 4811 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:08:11.362461 4811 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.154.129
I20260811 09:08:11.373319 4811 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:43515
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439250192320-29290-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439250192320-29290-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.129.keytab
--principal=kudu/127.28.154.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.154.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439250192320-29290-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.154.129
--webserver_port=0
--tserver_master_addrs=127.28.154.190:34691
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.154.129
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439250192320-29290-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:08:11.374760 4811 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:08:11.376132 4811 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:08:11.389568 4819 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:08:11.389482 4824 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:08:11.389590 4818 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:08:11.390659 4811 server_base.cc:1034] running on GCE node
I20260811 09:08:11.391418 4811 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:08:11.392776 4811 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:08:11.394053 4811 hybrid_clock.cc:648] HybridClock initialized: now 1786439291394003 us; error 38 us; skew 500 ppm
I20260811 09:08:11.394392 4811 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:08:11 dist-test-slave-xt4k krb5kdc[4618](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439291, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:08:11.397679 4811 init.cc:377] Logged in from keytab as kudu/127.28.154.129@KRBTEST.COM (short username kudu)
I20260811 09:08:11.398869 4811 fs_manager.cc:359] Metadata directory not provided
I20260811 09:08:11.398985 4811 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:08:11.399297 4811 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:08:11.402384 4811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439250192320-29290-0/minicluster-data/ts-0/data/instance:
uuid: "689189b7100d4fd4b05f2990370e97db"
format_stamp: "Formatted at 2026-08-11 09:08:11 on dist-test-slave-xt4k"
server_key: "ed2d412546b231e1ede04ed2b32ace73"
server_key_iv: "09ba64537ecae81e6d0d0b6a266726db"
server_key_version: "encryptionkey@0"
I20260811 09:08:11.403273 4811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439250192320-29290-0/minicluster-data/ts-0/wal/instance:
uuid: "689189b7100d4fd4b05f2990370e97db"
format_stamp: "Formatted at 2026-08-11 09:08:11 on dist-test-slave-xt4k"
server_key: "ed2d412546b231e1ede04ed2b32ace73"
server_key_iv: "09ba64537ecae81e6d0d0b6a266726db"
server_key_version: "encryptionkey@0"
I20260811 09:08:11.408671 4811 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 09:08:11.412310 4834 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:08:11.413893 4811 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 09:08:11.414065 4811 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439250192320-29290-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439250192320-29290-0/minicluster-data/ts-0/wal
uuid: "689189b7100d4fd4b05f2990370e97db"
format_stamp: "Formatted at 2026-08-11 09:08:11 on dist-test-slave-xt4k"
server_key: "ed2d412546b231e1ede04ed2b32ace73"
server_key_iv: "09ba64537ecae81e6d0d0b6a266726db"
server_key_version: "encryptionkey@0"
I20260811 09:08:11.414281 4811 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439250192320-29290-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439250192320-29290-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439250192320-29290-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:08:11.428958 4811 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:08:11.434067 4811 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:08:11.434387 4811 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:08:11.435490 4811 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:08:11.436972 4811 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:08:11.437075 4811 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:08:11.437186 4811 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:08:11.437258 4811 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:08:11.469015 4811 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.129:44273
I20260811 09:08:11.469071 5030 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.129:44273 every 8 connection(s)
I20260811 09:08:11.469906 4811 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439250192320-29290-0/minicluster-data/ts-0/data/info.pb
I20260811 09:08:11.474911 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 4811
I20260811 09:08:11.475114 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439250192320-29290-0/minicluster-data/ts-0/wal/instance
I20260811 09:08:11.475785 29290 external_mini_cluster.cc:1428] Setting key c7076b0f6c981bcbc7ca64f89900e459
Aug 11 09:08:11 dist-test-slave-xt4k krb5kdc[4618](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439291, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.129@KRBTEST.COM for kudu/127.28.154.190@KRBTEST.COM
I20260811 09:08:11.491673 5033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:11.472593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.129:49273 (local address 127.28.154.190:34691)
0811 09:08:11.473062 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:11.473066 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:11.474062 (+ 996us) server_negotiation.cc:407] Connection header received
0811 09:08:11.476038 (+ 1976us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:11.476045 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:11.476138 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:11.476270 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:11.478384 (+ 2114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:11.479446 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:11.481180 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:11.481708 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:11.481840 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:11.485362 (+ 3522us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:11.485369 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:11.485373 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:08:11.485412 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:11.488160 (+ 2748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:11.489068 (+ 908us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:11.489075 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:11.489079 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:11.489234 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:11.489804 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:11.489809 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:11.489811 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:11.490135 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:11.490283 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:11.491212 (+ 929us) server_negotiation.cc:299] Negotiation successful
0811 09:08:11.491402 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":261,"threads_started":1}
I20260811 09:08:11.493080 5036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:11.473154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:34691 (local address 127.28.154.129:49273)
0811 09:08:11.473904 (+ 750us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:11.473943 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:11.475732 (+ 1789us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:11.476577 (+ 845us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:11.476589 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:11.477343 (+ 754us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:08:11.478189 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:11.478217 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:11.479637 (+ 1420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:11.479642 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:11.481023 (+ 1381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:11.481037 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:11.481976 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:11.481982 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:11.482200 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:11.482885 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:08:11.482915 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:08:11.485122 (+ 2207us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:08:11.488321 (+ 3199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:08:11.488330 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:08:11.488369 (+ 39us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:08:11.488934 (+ 565us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:08:11.489361 (+ 427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:08:11.489366 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:08:11.489369 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:08:11.489611 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:08:11.490266 (+ 655us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:08:11.490275 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:08:11.490724 (+ 449us) client_negotiation.cc:769] Sending connection context
0811 09:08:11.491082 (+ 358us) client_negotiation.cc:240] Negotiation successful
0811 09:08:11.491638 (+ 556us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":301,"threads_started":1}
I20260811 09:08:11.494831 5031 heartbeater.cc:347] Connected to a master server at 127.28.154.190:34691
I20260811 09:08:11.495180 5031 heartbeater.cc:464] Registering TS with master...
I20260811 09:08:11.495999 5031 heartbeater.cc:511] Master 127.28.154.190:34691 requested a full tablet report, sending...
I20260811 09:08:11.498178 4700 ts_manager.cc:194] Registered new tserver with Master: 689189b7100d4fd4b05f2990370e97db (127.28.154.129:44273)
WARNING: no policy specified for kudu/127.28.154.130@KRBTEST.COM; defaulting to no policy
I20260811 09:08:11.499684 4700 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.154.129@KRBTEST.COM'} at 127.28.154.129:49273
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.154.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.154.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.130.keytab.
Entry for principal kudu/127.28.154.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.130.keytab.
I20260811 09:08:11.529548 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439250192320-29290-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439250192320-29290-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439250192320-29290-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439250192320-29290-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.154.130:0
--local_ip_for_outbound_sockets=127.28.154.130
--webserver_interface=127.28.154.130
--webserver_port=0
--tserver_master_addrs=127.28.154.190:34691
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.130.keytab
--principal=kudu/127.28.154.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:43515
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:08:11.865530 5039 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:08:11.865854 5039 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:08:11.865998 5039 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:08:11.873456 5039 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:08:11.873606 5039 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:08:11.873769 5039 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.154.130
I20260811 09:08:11.883975 5039 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:43515
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439250192320-29290-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439250192320-29290-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.130.keytab
--principal=kudu/127.28.154.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.154.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439250192320-29290-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.154.130
--webserver_port=0
--tserver_master_addrs=127.28.154.190:34691
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.154.130
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439250192320-29290-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:08:11.885407 5039 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:08:11.886751 5039 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:08:11.900059 5050 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:08:11.900074 5052 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:08:11.900113 5047 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:08:11.900851 5039 server_base.cc:1034] running on GCE node
I20260811 09:08:11.901564 5039 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:08:11.902621 5039 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:08:11.903843 5039 hybrid_clock.cc:648] HybridClock initialized: now 1786439291903796 us; error 39 us; skew 500 ppm
I20260811 09:08:11.904276 5039 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:08:11 dist-test-slave-xt4k krb5kdc[4618](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439291, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:08:11.907632 5039 init.cc:377] Logged in from keytab as kudu/127.28.154.130@KRBTEST.COM (short username kudu)
I20260811 09:08:11.908922 5039 fs_manager.cc:359] Metadata directory not provided
I20260811 09:08:11.909020 5039 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:08:11.909336 5039 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:08:11.912475 5039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439250192320-29290-0/minicluster-data/ts-1/data/instance:
uuid: "c8b06da9c2dd4a028e0038446f104cc8"
format_stamp: "Formatted at 2026-08-11 09:08:11 on dist-test-slave-xt4k"
server_key: "eed0e0c4c92721153779a85068e575ab"
server_key_iv: "081503105e128c2de542a75e3f482775"
server_key_version: "encryptionkey@0"
I20260811 09:08:11.913302 5039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439250192320-29290-0/minicluster-data/ts-1/wal/instance:
uuid: "c8b06da9c2dd4a028e0038446f104cc8"
format_stamp: "Formatted at 2026-08-11 09:08:11 on dist-test-slave-xt4k"
server_key: "eed0e0c4c92721153779a85068e575ab"
server_key_iv: "081503105e128c2de542a75e3f482775"
server_key_version: "encryptionkey@0"
I20260811 09:08:11.918779 5039 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260811 09:08:11.922551 5070 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:08:11.923976 5039 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 09:08:11.924160 5039 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439250192320-29290-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439250192320-29290-0/minicluster-data/ts-1/wal
uuid: "c8b06da9c2dd4a028e0038446f104cc8"
format_stamp: "Formatted at 2026-08-11 09:08:11 on dist-test-slave-xt4k"
server_key: "eed0e0c4c92721153779a85068e575ab"
server_key_iv: "081503105e128c2de542a75e3f482775"
server_key_version: "encryptionkey@0"
I20260811 09:08:11.924424 5039 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439250192320-29290-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439250192320-29290-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439250192320-29290-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:08:11.936282 5039 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:08:11.940562 5039 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:08:11.940828 5039 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:08:11.941834 5039 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:08:11.943252 5039 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:08:11.943351 5039 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:08:11.943461 5039 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:08:11.943540 5039 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:08:11.976531 5039 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.130:34479
I20260811 09:08:11.976603 5275 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.130:34479 every 8 connection(s)
I20260811 09:08:11.977464 5039 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439250192320-29290-0/minicluster-data/ts-1/data/info.pb
I20260811 09:08:11.983199 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 5039
I20260811 09:08:11.983393 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439250192320-29290-0/minicluster-data/ts-1/wal/instance
I20260811 09:08:11.983937 29290 external_mini_cluster.cc:1428] Setting key c4facaeee30d0b3f1d53827a42cf5f81
Aug 11 09:08:11 dist-test-slave-xt4k krb5kdc[4618](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439291, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.130@KRBTEST.COM for kudu/127.28.154.190@KRBTEST.COM
I20260811 09:08:12.000660 5033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:11.979921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.130:53229 (local address 127.28.154.190:34691)
0811 09:08:11.980116 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:11.980121 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:11.981367 (+ 1246us) server_negotiation.cc:407] Connection header received
0811 09:08:11.983497 (+ 2130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:11.983502 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:11.983584 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:11.983683 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:11.985693 (+ 2010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:11.986594 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:11.988985 (+ 2391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:11.989503 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:11.989631 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:11.994117 (+ 4486us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:11.994125 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:11.994130 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:08:11.994178 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:11.997109 (+ 2931us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:11.997952 (+ 843us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:11.997957 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:11.997959 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:11.998110 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:11.998764 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:11.998768 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:11.998770 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:11.999076 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:11.999223 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:12.000000 (+ 777us) server_negotiation.cc:299] Negotiation successful
0811 09:08:12.000261 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 09:08:12.001711 5281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:11.980426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:34691 (local address 127.28.154.130:53229)
0811 09:08:11.981190 (+ 764us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:11.981232 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:11.983137 (+ 1905us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:11.983965 (+ 828us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:11.983978 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:11.984765 (+ 787us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:08:11.985497 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:11.985517 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:11.986791 (+ 1274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:11.986797 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:11.988805 (+ 2008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:11.988822 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:11.989738 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:11.989756 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:11.990038 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:11.990807 (+ 769us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:08:11.990839 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:08:11.993701 (+ 2862us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:08:11.997303 (+ 3602us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:08:11.997312 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:08:11.997329 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:08:11.997810 (+ 481us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:08:11.998236 (+ 426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:08:11.998240 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:08:11.998243 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:08:11.998614 (+ 371us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:08:11.999214 (+ 600us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:08:11.999222 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:08:11.999602 (+ 380us) client_negotiation.cc:769] Sending connection context
0811 09:08:11.999871 (+ 269us) client_negotiation.cc:240] Negotiation successful
0811 09:08:12.000482 (+ 611us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":306,"threads_started":1}
I20260811 09:08:12.003402 5278 heartbeater.cc:347] Connected to a master server at 127.28.154.190:34691
I20260811 09:08:12.003748 5278 heartbeater.cc:464] Registering TS with master...
I20260811 09:08:12.004621 5278 heartbeater.cc:511] Master 127.28.154.190:34691 requested a full tablet report, sending...
I20260811 09:08:12.006322 4700 ts_manager.cc:194] Registered new tserver with Master: c8b06da9c2dd4a028e0038446f104cc8 (127.28.154.130:34479)
I20260811 09:08:12.007522 4700 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.154.130@KRBTEST.COM'} at 127.28.154.130:53229
WARNING: no policy specified for kudu/127.28.154.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.154.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.154.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.131.keytab.
Entry for principal kudu/127.28.154.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.131.keytab.
I20260811 09:08:12.039911 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439250192320-29290-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439250192320-29290-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439250192320-29290-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439250192320-29290-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.154.131:0
--local_ip_for_outbound_sockets=127.28.154.131
--webserver_interface=127.28.154.131
--webserver_port=0
--tserver_master_addrs=127.28.154.190:34691
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.131.keytab
--principal=kudu/127.28.154.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:43515
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:08:12.391822 5290 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:08:12.392130 5290 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:08:12.392254 5290 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:08:12.399688 5290 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:08:12.399820 5290 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:08:12.399974 5290 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.154.131
I20260811 09:08:12.410631 5290 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:43515
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439250192320-29290-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439250192320-29290-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.131.keytab
--principal=kudu/127.28.154.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.154.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439250192320-29290-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.154.131
--webserver_port=0
--tserver_master_addrs=127.28.154.190:34691
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.154.131
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439250192320-29290-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:08:12.412055 5290 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:08:12.413440 5290 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:08:12.426658 5296 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:08:12.426683 5299 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:08:12.426632 5302 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:08:12.427731 5290 server_base.cc:1034] running on GCE node
I20260811 09:08:12.428507 5290 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:08:12.429569 5290 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:08:12.430790 5290 hybrid_clock.cc:648] HybridClock initialized: now 1786439292430723 us; error 62 us; skew 500 ppm
I20260811 09:08:12.431123 5290 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:08:12 dist-test-slave-xt4k krb5kdc[4618](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439292, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:08:12.434615 5290 init.cc:377] Logged in from keytab as kudu/127.28.154.131@KRBTEST.COM (short username kudu)
I20260811 09:08:12.435832 5290 fs_manager.cc:359] Metadata directory not provided
I20260811 09:08:12.435942 5290 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:08:12.436265 5290 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:08:12.439363 5290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439250192320-29290-0/minicluster-data/ts-2/data/instance:
uuid: "1343406c45a64eb4b66f49e58bd371fd"
format_stamp: "Formatted at 2026-08-11 09:08:12 on dist-test-slave-xt4k"
server_key: "2f448e6508f7e7d1d3c0dbd40449b34c"
server_key_iv: "ec836cb7a3b1232477166f6622dd8055"
server_key_version: "encryptionkey@0"
I20260811 09:08:12.440199 5290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439250192320-29290-0/minicluster-data/ts-2/wal/instance:
uuid: "1343406c45a64eb4b66f49e58bd371fd"
format_stamp: "Formatted at 2026-08-11 09:08:12 on dist-test-slave-xt4k"
server_key: "2f448e6508f7e7d1d3c0dbd40449b34c"
server_key_iv: "ec836cb7a3b1232477166f6622dd8055"
server_key_version: "encryptionkey@0"
I20260811 09:08:12.445642 5290 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 09:08:12.449379 5312 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:08:12.450814 5290 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 09:08:12.451032 5290 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439250192320-29290-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439250192320-29290-0/minicluster-data/ts-2/wal
uuid: "1343406c45a64eb4b66f49e58bd371fd"
format_stamp: "Formatted at 2026-08-11 09:08:12 on dist-test-slave-xt4k"
server_key: "2f448e6508f7e7d1d3c0dbd40449b34c"
server_key_iv: "ec836cb7a3b1232477166f6622dd8055"
server_key_version: "encryptionkey@0"
I20260811 09:08:12.451258 5290 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439250192320-29290-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439250192320-29290-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439250192320-29290-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:08:12.479362 5290 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:08:12.483888 5290 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:08:12.484192 5290 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:08:12.485275 5290 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:08:12.486701 5290 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:08:12.486801 5290 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:08:12.486912 5290 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:08:12.486991 5290 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:08:12.503050 5031 heartbeater.cc:503] Master 127.28.154.190:34691 was elected leader, sending a full tablet report...
I20260811 09:08:12.522912 5290 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.131:35779
I20260811 09:08:12.523031 5511 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.131:35779 every 8 connection(s)
I20260811 09:08:12.523873 5290 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439250192320-29290-0/minicluster-data/ts-2/data/info.pb
I20260811 09:08:12.525348 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 5290
I20260811 09:08:12.525533 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439250192320-29290-0/minicluster-data/ts-2/wal/instance
I20260811 09:08:12.526017 29290 external_mini_cluster.cc:1428] Setting key 056ea44f22ddcdfbf9eaf1fe2e639966
Aug 11 09:08:12 dist-test-slave-xt4k krb5kdc[4618](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439292, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.131@KRBTEST.COM for kudu/127.28.154.190@KRBTEST.COM
I20260811 09:08:12.548424 5514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:12.527930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.131:46571 (local address 127.28.154.190:34691)
0811 09:08:12.528547 (+ 617us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:12.528555 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:12.529279 (+ 724us) server_negotiation.cc:407] Connection header received
0811 09:08:12.531599 (+ 2320us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:12.531604 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:12.531692 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:12.531787 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:12.533583 (+ 1796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:12.534361 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:12.536026 (+ 1665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:12.536525 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:12.536640 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:12.540411 (+ 3771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:12.540416 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:12.540420 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:08:12.540456 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:12.544423 (+ 3967us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:12.545404 (+ 981us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:12.545409 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:12.545411 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:12.545580 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:12.546080 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:12.546085 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:12.546087 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:12.546406 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:12.546556 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:12.547480 (+ 924us) server_negotiation.cc:299] Negotiation successful
0811 09:08:12.547908 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":471,"thread_start_us":293,"threads_started":1}
I20260811 09:08:12.549319 5515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:12.527751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:34691 (local address 127.28.154.131:46571)
0811 09:08:12.529003 (+ 1252us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:12.529124 (+ 121us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:12.531311 (+ 2187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:12.532059 (+ 748us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:12.532069 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:12.532685 (+ 616us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:08:12.533372 (+ 687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:12.533393 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:12.534536 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:12.534540 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:12.535870 (+ 1330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:12.535885 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:12.536773 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:12.536782 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:12.537022 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:12.537759 (+ 737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:08:12.537783 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:08:12.540132 (+ 2349us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:08:12.544603 (+ 4471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:08:12.544614 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:08:12.544637 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:08:12.545272 (+ 635us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:08:12.545721 (+ 449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:08:12.545726 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:08:12.545728 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:08:12.545957 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:08:12.546568 (+ 611us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:08:12.546576 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:08:12.547092 (+ 516us) client_negotiation.cc:769] Sending connection context
0811 09:08:12.547342 (+ 250us) client_negotiation.cc:240] Negotiation successful
0811 09:08:12.547801 (+ 459us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":743,"thread_start_us":523,"threads_started":1}
I20260811 09:08:12.551276 5512 heartbeater.cc:347] Connected to a master server at 127.28.154.190:34691
I20260811 09:08:12.551602 5512 heartbeater.cc:464] Registering TS with master...
I20260811 09:08:12.552259 5512 heartbeater.cc:511] Master 127.28.154.190:34691 requested a full tablet report, sending...
I20260811 09:08:12.553794 4700 ts_manager.cc:194] Registered new tserver with Master: 1343406c45a64eb4b66f49e58bd371fd (127.28.154.131:35779)
I20260811 09:08:12.554627 4700 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.154.131@KRBTEST.COM'} at 127.28.154.131:46571
I20260811 09:08:12.557401 29290 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 09:08:12.576818 5514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:12.563048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43350 (local address 127.28.154.190:34691)
0811 09:08:12.563284 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:12.563291 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:12.563597 (+ 306us) server_negotiation.cc:407] Connection header received
0811 09:08:12.563978 (+ 381us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:12.563982 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:12.564056 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:12.564152 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:12.565675 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:12.566432 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:12.568045 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:12.568579 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:12.568705 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:12.571166 (+ 2461us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:12.571172 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:12.571175 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:08:12.571210 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:12.574011 (+ 2801us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:12.574735 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:12.574740 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:12.574743 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:12.574874 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:12.575240 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:12.575245 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:12.575247 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:12.575601 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:12.575846 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:12.576209 (+ 363us) server_negotiation.cc:299] Negotiation successful
0811 09:08:12.576479 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260811 09:08:12.581862 4700 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43350:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 09:08:12.589279 4700 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 09:08:12.613540 5532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:12.603808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59220 (local address 127.28.154.130:34479)
0811 09:08:12.604585 (+ 777us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:12.604595 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:12.605160 (+ 565us) server_negotiation.cc:407] Connection header received
0811 09:08:12.605502 (+ 342us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:12.605511 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:12.605778 (+ 267us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:12.605974 (+ 196us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:08:12.606906 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:12.608372 (+ 1466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:12.610573 (+ 2201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:12.612055 (+ 1482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:12.612220 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:12.612432 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:12.612980 (+ 548us) server_negotiation.cc:299] Negotiation successful
0811 09:08:12.613171 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":591,"thread_start_us":765,"threads_started":1}
I20260811 09:08:12.613524 5537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:12.603775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.130:34479 (local address 127.0.0.1:59220)
0811 09:08:12.604984 (+ 1209us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:12.605022 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:12.605251 (+ 229us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:12.606261 (+ 1010us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:12.606268 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:12.606300 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:08:12.606698 (+ 398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:12.606710 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:12.608521 (+ 1811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:12.608527 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:12.610363 (+ 1836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:12.610375 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:12.612410 (+ 2035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:12.612423 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:12.612700 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:12.612757 (+ 57us) client_negotiation.cc:769] Sending connection context
0811 09:08:12.612912 (+ 155us) client_negotiation.cc:240] Negotiation successful
0811 09:08:12.613120 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":958,"thread_start_us":1842,"threads_started":1}
I20260811 09:08:12.615002 5539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:12.604400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37930 (local address 127.28.154.131:35779)
0811 09:08:12.605023 (+ 623us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:12.605029 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:12.605733 (+ 704us) server_negotiation.cc:407] Connection header received
0811 09:08:12.605834 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:12.605841 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:12.606056 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:12.606234 (+ 178us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:08:12.607539 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:12.609078 (+ 1539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:12.611755 (+ 2677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:12.613203 (+ 1448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:12.613397 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:12.613569 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:12.614491 (+ 922us) server_negotiation.cc:299] Negotiation successful
0811 09:08:12.614668 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":477,"thread_start_us":357,"threads_started":1}
I20260811 09:08:12.615016 5541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:12.605528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.129:44273 (local address 127.0.0.1:35082)
0811 09:08:12.606159 (+ 631us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:12.606180 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:12.606295 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:12.608090 (+ 1795us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:12.608095 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:12.608120 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:08:12.608562 (+ 442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:12.608574 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:12.610302 (+ 1728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:12.610311 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:12.612497 (+ 2186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:12.612513 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:12.614206 (+ 1693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:12.614212 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:12.614389 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:12.614418 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 09:08:12.614508 (+ 90us) client_negotiation.cc:240] Negotiation successful
0811 09:08:12.614650 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":515,"mutex_wait_us":3,"thread_start_us":599,"threads_started":1}
I20260811 09:08:12.615746 5542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:12.605936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35082 (local address 127.28.154.129:44273)
0811 09:08:12.607520 (+ 1584us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:12.607528 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:12.607554 (+ 26us) server_negotiation.cc:407] Connection header received
0811 09:08:12.607647 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:12.607653 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:12.607878 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:12.608086 (+ 208us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:08:12.608751 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:12.610142 (+ 1391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:12.612653 (+ 2511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:12.614000 (+ 1347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:12.614213 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:12.614375 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:12.615234 (+ 859us) server_negotiation.cc:299] Negotiation successful
0811 09:08:12.615401 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1420,"thread_start_us":1262,"threads_started":1}
I20260811 09:08:12.615798 5538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:12.603904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.131:35779 (local address 127.0.0.1:37930)
0811 09:08:12.605490 (+ 1586us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:12.605507 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:12.605622 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:12.606512 (+ 890us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:12.606516 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:12.606537 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:08:12.606962 (+ 425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:12.606975 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:12.609228 (+ 2253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:12.609233 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:12.611590 (+ 2357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:12.611603 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:12.613729 (+ 2126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:12.613736 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:12.613914 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:12.613947 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 09:08:12.614035 (+ 88us) client_negotiation.cc:240] Negotiation successful
0811 09:08:12.614189 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1446,"mutex_wait_us":2,"thread_start_us":716,"threads_started":1}
I20260811 09:08:12.618316 5159 tablet_service.cc:1467] Processing CreateTablet for tablet e1eaac31db1f498c8731c682a8939b6a (DEFAULT_TABLE table=test-table [id=c5fb6454b4ca415689977d5f57a1c969]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:08:12.619207 5395 tablet_service.cc:1467] Processing CreateTablet for tablet e1eaac31db1f498c8731c682a8939b6a (DEFAULT_TABLE table=test-table [id=c5fb6454b4ca415689977d5f57a1c969]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:08:12.620128 5159 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1eaac31db1f498c8731c682a8939b6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:08:12.619982 4926 tablet_service.cc:1467] Processing CreateTablet for tablet e1eaac31db1f498c8731c682a8939b6a (DEFAULT_TABLE table=test-table [id=c5fb6454b4ca415689977d5f57a1c969]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:08:12.620893 5395 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1eaac31db1f498c8731c682a8939b6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:08:12.621398 4926 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1eaac31db1f498c8731c682a8939b6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:08:12.632937 5543 tablet_bootstrap.cc:492] T e1eaac31db1f498c8731c682a8939b6a P 1343406c45a64eb4b66f49e58bd371fd: Bootstrap starting.
I20260811 09:08:12.636603 5544 tablet_bootstrap.cc:492] T e1eaac31db1f498c8731c682a8939b6a P c8b06da9c2dd4a028e0038446f104cc8: Bootstrap starting.
I20260811 09:08:12.637418 5548 tablet_bootstrap.cc:492] T e1eaac31db1f498c8731c682a8939b6a P 689189b7100d4fd4b05f2990370e97db: Bootstrap starting.
I20260811 09:08:12.637559 5543 tablet_bootstrap.cc:654] T e1eaac31db1f498c8731c682a8939b6a P 1343406c45a64eb4b66f49e58bd371fd: Neither blocks nor log segments found. Creating new log.
I20260811 09:08:12.638923 5543 log.cc:824] T e1eaac31db1f498c8731c682a8939b6a P 1343406c45a64eb4b66f49e58bd371fd: Log is configured to *not* fsync() on all Append() calls
I20260811 09:08:12.640609 5544 tablet_bootstrap.cc:654] T e1eaac31db1f498c8731c682a8939b6a P c8b06da9c2dd4a028e0038446f104cc8: Neither blocks nor log segments found. Creating new log.
I20260811 09:08:12.640694 5548 tablet_bootstrap.cc:654] T e1eaac31db1f498c8731c682a8939b6a P 689189b7100d4fd4b05f2990370e97db: Neither blocks nor log segments found. Creating new log.
I20260811 09:08:12.641563 5544 log.cc:824] T e1eaac31db1f498c8731c682a8939b6a P c8b06da9c2dd4a028e0038446f104cc8: Log is configured to *not* fsync() on all Append() calls
I20260811 09:08:12.641898 5548 log.cc:824] T e1eaac31db1f498c8731c682a8939b6a P 689189b7100d4fd4b05f2990370e97db: Log is configured to *not* fsync() on all Append() calls
I20260811 09:08:12.642423 5543 tablet_bootstrap.cc:492] T e1eaac31db1f498c8731c682a8939b6a P 1343406c45a64eb4b66f49e58bd371fd: No bootstrap required, opened a new log
I20260811 09:08:12.642799 5543 ts_tablet_manager.cc:1397] T e1eaac31db1f498c8731c682a8939b6a P 1343406c45a64eb4b66f49e58bd371fd: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.008s
I20260811 09:08:12.644850 5544 tablet_bootstrap.cc:492] T e1eaac31db1f498c8731c682a8939b6a P c8b06da9c2dd4a028e0038446f104cc8: No bootstrap required, opened a new log
I20260811 09:08:12.645102 5544 ts_tablet_manager.cc:1397] T e1eaac31db1f498c8731c682a8939b6a P c8b06da9c2dd4a028e0038446f104cc8: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260811 09:08:12.650560 5543 raft_consensus.cc:348] T e1eaac31db1f498c8731c682a8939b6a P 1343406c45a64eb4b66f49e58bd371fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8b06da9c2dd4a028e0038446f104cc8" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34479 } } peers { permanent_uuid: "1343406c45a64eb4b66f49e58bd371fd" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 35779 } } peers { permanent_uuid: "689189b7100d4fd4b05f2990370e97db" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 44273 } }
I20260811 09:08:12.650944 5543 raft_consensus.cc:374] T e1eaac31db1f498c8731c682a8939b6a P 1343406c45a64eb4b66f49e58bd371fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:08:12.651072 5543 raft_consensus.cc:729] T e1eaac31db1f498c8731c682a8939b6a P 1343406c45a64eb4b66f49e58bd371fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1343406c45a64eb4b66f49e58bd371fd, State: Initialized, Role: FOLLOWER
I20260811 09:08:12.651746 5543 consensus_queue.cc:260] T e1eaac31db1f498c8731c682a8939b6a P 1343406c45a64eb4b66f49e58bd371fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8b06da9c2dd4a028e0038446f104cc8" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34479 } } peers { permanent_uuid: "1343406c45a64eb4b66f49e58bd371fd" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 35779 } } peers { permanent_uuid: "689189b7100d4fd4b05f2990370e97db" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 44273 } }
I20260811 09:08:12.652649 5544 raft_consensus.cc:348] T e1eaac31db1f498c8731c682a8939b6a P c8b06da9c2dd4a028e0038446f104cc8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8b06da9c2dd4a028e0038446f104cc8" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34479 } } peers { permanent_uuid: "1343406c45a64eb4b66f49e58bd371fd" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 35779 } } peers { permanent_uuid: "689189b7100d4fd4b05f2990370e97db" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 44273 } }
I20260811 09:08:12.652987 5512 heartbeater.cc:503] Master 127.28.154.190:34691 was elected leader, sending a full tablet report...
I20260811 09:08:12.653198 5544 raft_consensus.cc:374] T e1eaac31db1f498c8731c682a8939b6a P c8b06da9c2dd4a028e0038446f104cc8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:08:12.653350 5544 raft_consensus.cc:729] T e1eaac31db1f498c8731c682a8939b6a P c8b06da9c2dd4a028e0038446f104cc8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8b06da9c2dd4a028e0038446f104cc8, State: Initialized, Role: FOLLOWER
I20260811 09:08:12.653702 5543 ts_tablet_manager.cc:1428] T e1eaac31db1f498c8731c682a8939b6a P 1343406c45a64eb4b66f49e58bd371fd: Time spent starting tablet: real 0.011s user 0.008s sys 0.001s
I20260811 09:08:12.654070 5548 tablet_bootstrap.cc:492] T e1eaac31db1f498c8731c682a8939b6a P 689189b7100d4fd4b05f2990370e97db: No bootstrap required, opened a new log
I20260811 09:08:12.654155 5544 consensus_queue.cc:260] T e1eaac31db1f498c8731c682a8939b6a P c8b06da9c2dd4a028e0038446f104cc8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8b06da9c2dd4a028e0038446f104cc8" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34479 } } peers { permanent_uuid: "1343406c45a64eb4b66f49e58bd371fd" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 35779 } } peers { permanent_uuid: "689189b7100d4fd4b05f2990370e97db" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 44273 } }
I20260811 09:08:12.654750 5548 ts_tablet_manager.cc:1397] T e1eaac31db1f498c8731c682a8939b6a P 689189b7100d4fd4b05f2990370e97db: Time spent bootstrapping tablet: real 0.018s user 0.008s sys 0.000s
I20260811 09:08:12.655548 5278 heartbeater.cc:503] Master 127.28.154.190:34691 was elected leader, sending a full tablet report...
I20260811 09:08:12.656872 5544 ts_tablet_manager.cc:1428] T e1eaac31db1f498c8731c682a8939b6a P c8b06da9c2dd4a028e0038446f104cc8: Time spent starting tablet: real 0.012s user 0.008s sys 0.004s
I20260811 09:08:12.664327 5548 raft_consensus.cc:348] T e1eaac31db1f498c8731c682a8939b6a P 689189b7100d4fd4b05f2990370e97db [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8b06da9c2dd4a028e0038446f104cc8" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34479 } } peers { permanent_uuid: "1343406c45a64eb4b66f49e58bd371fd" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 35779 } } peers { permanent_uuid: "689189b7100d4fd4b05f2990370e97db" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 44273 } }
I20260811 09:08:12.664798 5548 raft_consensus.cc:374] T e1eaac31db1f498c8731c682a8939b6a P 689189b7100d4fd4b05f2990370e97db [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:08:12.664901 5548 raft_consensus.cc:729] T e1eaac31db1f498c8731c682a8939b6a P 689189b7100d4fd4b05f2990370e97db [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 689189b7100d4fd4b05f2990370e97db, State: Initialized, Role: FOLLOWER
I20260811 09:08:12.665503 5548 consensus_queue.cc:260] T e1eaac31db1f498c8731c682a8939b6a P 689189b7100d4fd4b05f2990370e97db [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8b06da9c2dd4a028e0038446f104cc8" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34479 } } peers { permanent_uuid: "1343406c45a64eb4b66f49e58bd371fd" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 35779 } } peers { permanent_uuid: "689189b7100d4fd4b05f2990370e97db" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 44273 } }
I20260811 09:08:12.667068 5548 ts_tablet_manager.cc:1428] T e1eaac31db1f498c8731c682a8939b6a P 689189b7100d4fd4b05f2990370e97db: Time spent starting tablet: real 0.012s user 0.011s sys 0.000s
W20260811 09:08:12.722154 5032 tablet.cc:2369] T e1eaac31db1f498c8731c682a8939b6a P 689189b7100d4fd4b05f2990370e97db: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 09:08:12.729292 5279 tablet.cc:2369] T e1eaac31db1f498c8731c682a8939b6a P c8b06da9c2dd4a028e0038446f104cc8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 09:08:12.775554 5513 tablet.cc:2369] T e1eaac31db1f498c8731c682a8939b6a P 1343406c45a64eb4b66f49e58bd371fd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 09:08:12.820312 5561 raft_consensus.cc:482] T e1eaac31db1f498c8731c682a8939b6a P 689189b7100d4fd4b05f2990370e97db [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:08:12.820657 5561 raft_consensus.cc:504] T e1eaac31db1f498c8731c682a8939b6a P 689189b7100d4fd4b05f2990370e97db [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8b06da9c2dd4a028e0038446f104cc8" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34479 } } peers { permanent_uuid: "1343406c45a64eb4b66f49e58bd371fd" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 35779 } } peers { permanent_uuid: "689189b7100d4fd4b05f2990370e97db" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 44273 } }
I20260811 09:08:12.822429 5561 leader_election.cc:288] T e1eaac31db1f498c8731c682a8939b6a P 689189b7100d4fd4b05f2990370e97db [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c8b06da9c2dd4a028e0038446f104cc8 (127.28.154.130:34479), 1343406c45a64eb4b66f49e58bd371fd (127.28.154.131:35779)
I20260811 09:08:12.829696 5532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:12.823116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.129:43749 (local address 127.28.154.130:34479)
0811 09:08:12.823321 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:12.823327 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:12.824160 (+ 833us) server_negotiation.cc:407] Connection header received
0811 09:08:12.824269 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:12.824274 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:12.824370 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:12.824463 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:08:12.825141 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:12.825886 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:12.827796 (+ 1910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:12.828681 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:12.828820 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:12.828877 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:12.829297 (+ 420us) server_negotiation.cc:299] Negotiation successful
0811 09:08:12.829428 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 09:08:12.829707 5539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:12.822965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.129:48829 (local address 127.28.154.131:35779)
0811 09:08:12.823167 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:12.823173 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:12.823911 (+ 738us) server_negotiation.cc:407] Connection header received
0811 09:08:12.824266 (+ 355us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:12.824271 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:12.824330 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:12.824436 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:08:12.825154 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:12.825868 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:12.827796 (+ 1928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:12.828820 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:12.828993 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:12.829078 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:12.829307 (+ 229us) server_negotiation.cc:299] Negotiation successful
0811 09:08:12.829470 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 09:08:12.829865 5563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:12.823009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.130:34479 (local address 127.28.154.129:43749)
0811 09:08:12.823915 (+ 906us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:12.823933 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:12.824059 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:12.824501 (+ 442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:12.824505 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:12.824528 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:08:12.824959 (+ 431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:12.824975 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:12.826024 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:12.826027 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:12.827631 (+ 1604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:12.827648 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:12.828943 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:12.828950 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:12.829138 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:12.829163 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 09:08:12.829245 (+ 82us) client_negotiation.cc:240] Negotiation successful
0811 09:08:12.829456 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":795,"mutex_wait_us":39,"thread_start_us":576,"threads_started":1}
I20260811 09:08:12.829840 5562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:12.822964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.131:35779 (local address 127.28.154.129:48829)
0811 09:08:12.823797 (+ 833us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:12.823813 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:12.823978 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:12.824460 (+ 482us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:12.824465 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:12.824492 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:08:12.825004 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:12.825018 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:12.826003 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:12.826009 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:12.827633 (+ 1624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:12.827646 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:12.829029 (+ 1383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:12.829034 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:12.829159 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:12.829182 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 09:08:12.829270 (+ 88us) client_negotiation.cc:240] Negotiation successful
0811 09:08:12.829405 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":706,"mutex_wait_us":23,"thread_start_us":407,"threads_started":1}
I20260811 09:08:12.831089 5433 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1eaac31db1f498c8731c682a8939b6a" candidate_uuid: "689189b7100d4fd4b05f2990370e97db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1343406c45a64eb4b66f49e58bd371fd" is_pre_election: true
I20260811 09:08:12.831087 5197 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1eaac31db1f498c8731c682a8939b6a" candidate_uuid: "689189b7100d4fd4b05f2990370e97db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8b06da9c2dd4a028e0038446f104cc8" is_pre_election: true
I20260811 09:08:12.831622 5433 raft_consensus.cc:2454] T e1eaac31db1f498c8731c682a8939b6a P 1343406c45a64eb4b66f49e58bd371fd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 689189b7100d4fd4b05f2990370e97db in term 0.
I20260811 09:08:12.831621 5197 raft_consensus.cc:2454] T e1eaac31db1f498c8731c682a8939b6a P c8b06da9c2dd4a028e0038446f104cc8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 689189b7100d4fd4b05f2990370e97db in term 0.
I20260811 09:08:12.832312 4835 leader_election.cc:302] T e1eaac31db1f498c8731c682a8939b6a P 689189b7100d4fd4b05f2990370e97db [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 689189b7100d4fd4b05f2990370e97db, c8b06da9c2dd4a028e0038446f104cc8; no voters:
I20260811 09:08:12.832793 5561 raft_consensus.cc:2781] T e1eaac31db1f498c8731c682a8939b6a P 689189b7100d4fd4b05f2990370e97db [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 09:08:12.832969 5561 raft_consensus.cc:482] T e1eaac31db1f498c8731c682a8939b6a P 689189b7100d4fd4b05f2990370e97db [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 09:08:12.833078 5561 raft_consensus.cc:3037] T e1eaac31db1f498c8731c682a8939b6a P 689189b7100d4fd4b05f2990370e97db [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:08:12.835290 5561 raft_consensus.cc:504] T e1eaac31db1f498c8731c682a8939b6a P 689189b7100d4fd4b05f2990370e97db [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8b06da9c2dd4a028e0038446f104cc8" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34479 } } peers { permanent_uuid: "1343406c45a64eb4b66f49e58bd371fd" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 35779 } } peers { permanent_uuid: "689189b7100d4fd4b05f2990370e97db" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 44273 } }
I20260811 09:08:12.835948 5561 leader_election.cc:288] T e1eaac31db1f498c8731c682a8939b6a P 689189b7100d4fd4b05f2990370e97db [CANDIDATE]: Term 1 election: Requested vote from peers c8b06da9c2dd4a028e0038446f104cc8 (127.28.154.130:34479), 1343406c45a64eb4b66f49e58bd371fd (127.28.154.131:35779)
I20260811 09:08:12.836504 5197 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1eaac31db1f498c8731c682a8939b6a" candidate_uuid: "689189b7100d4fd4b05f2990370e97db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c8b06da9c2dd4a028e0038446f104cc8"
I20260811 09:08:12.836535 5433 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1eaac31db1f498c8731c682a8939b6a" candidate_uuid: "689189b7100d4fd4b05f2990370e97db" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1343406c45a64eb4b66f49e58bd371fd"
I20260811 09:08:12.836748 5433 raft_consensus.cc:3037] T e1eaac31db1f498c8731c682a8939b6a P 1343406c45a64eb4b66f49e58bd371fd [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:08:12.836748 5197 raft_consensus.cc:3037] T e1eaac31db1f498c8731c682a8939b6a P c8b06da9c2dd4a028e0038446f104cc8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:08:12.839987 5197 raft_consensus.cc:2454] T e1eaac31db1f498c8731c682a8939b6a P c8b06da9c2dd4a028e0038446f104cc8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 689189b7100d4fd4b05f2990370e97db in term 1.
I20260811 09:08:12.839987 5433 raft_consensus.cc:2454] T e1eaac31db1f498c8731c682a8939b6a P 1343406c45a64eb4b66f49e58bd371fd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 689189b7100d4fd4b05f2990370e97db in term 1.
I20260811 09:08:12.840545 4844 leader_election.cc:302] T e1eaac31db1f498c8731c682a8939b6a P 689189b7100d4fd4b05f2990370e97db [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1343406c45a64eb4b66f49e58bd371fd, 689189b7100d4fd4b05f2990370e97db; no voters:
I20260811 09:08:12.840863 5561 raft_consensus.cc:2781] T e1eaac31db1f498c8731c682a8939b6a P 689189b7100d4fd4b05f2990370e97db [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:08:12.841351 5561 raft_consensus.cc:686] T e1eaac31db1f498c8731c682a8939b6a P 689189b7100d4fd4b05f2990370e97db [term 1 LEADER]: Becoming Leader. State: Replica: 689189b7100d4fd4b05f2990370e97db, State: Running, Role: LEADER
I20260811 09:08:12.841917 5561 consensus_queue.cc:237] T e1eaac31db1f498c8731c682a8939b6a P 689189b7100d4fd4b05f2990370e97db [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8b06da9c2dd4a028e0038446f104cc8" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34479 } } peers { permanent_uuid: "1343406c45a64eb4b66f49e58bd371fd" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 35779 } } peers { permanent_uuid: "689189b7100d4fd4b05f2990370e97db" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 44273 } }
I20260811 09:08:12.847374 4700 catalog_manager.cc:5495] T e1eaac31db1f498c8731c682a8939b6a P 689189b7100d4fd4b05f2990370e97db reported cstate change: term changed from 0 to 1, leader changed from <none> to 689189b7100d4fd4b05f2990370e97db (127.28.154.129). New cstate: current_term: 1 leader_uuid: "689189b7100d4fd4b05f2990370e97db" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c8b06da9c2dd4a028e0038446f104cc8" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34479 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1343406c45a64eb4b66f49e58bd371fd" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 35779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "689189b7100d4fd4b05f2990370e97db" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 44273 } health_report { overall_health: HEALTHY } } }
I20260811 09:08:12.914286 5542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:12.908412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35088 (local address 127.28.154.129:44273)
0811 09:08:12.908644 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:12.908649 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:12.908789 (+ 140us) server_negotiation.cc:407] Connection header received
0811 09:08:12.909167 (+ 378us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:12.909172 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:12.909287 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:12.909426 (+ 139us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:08:12.909836 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:12.910649 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:12.912060 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:12.912518 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:12.912611 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:12.913120 (+ 509us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:08:12.913705 (+ 585us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:08:12.913816 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:12.913914 (+ 98us) server_negotiation.cc:299] Negotiation successful
0811 09:08:12.914012 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 09:08:12.922947 5433 raft_consensus.cc:1261] T e1eaac31db1f498c8731c682a8939b6a P 1343406c45a64eb4b66f49e58bd371fd [term 1 FOLLOWER]: Refusing update from remote peer 689189b7100d4fd4b05f2990370e97db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:08:12.922947 5197 raft_consensus.cc:1261] T e1eaac31db1f498c8731c682a8939b6a P c8b06da9c2dd4a028e0038446f104cc8 [term 1 FOLLOWER]: Refusing update from remote peer 689189b7100d4fd4b05f2990370e97db: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:08:12.923993 5564 consensus_queue.cc:1035] T e1eaac31db1f498c8731c682a8939b6a P 689189b7100d4fd4b05f2990370e97db [LEADER]: Connected to new peer: Peer: permanent_uuid: "1343406c45a64eb4b66f49e58bd371fd" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 35779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:08:12.924378 5561 consensus_queue.cc:1035] T e1eaac31db1f498c8731c682a8939b6a P 689189b7100d4fd4b05f2990370e97db [LEADER]: Connected to new peer: Peer: permanent_uuid: "c8b06da9c2dd4a028e0038446f104cc8" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:08:12.937552 5575 mvcc.cc:205] Tried to move back new op lower bound from 7317255343802068992 to 7317255343485448192. Current Snapshot: MvccSnapshot[applied={T|T < 7317255343802068992}]
I20260811 09:08:12.940322 5576 mvcc.cc:205] Tried to move back new op lower bound from 7317255343802068992 to 7317255343485448192. Current Snapshot: MvccSnapshot[applied={T|T < 7317255343802068992}]
I20260811 09:08:12.945359 5574 mvcc.cc:205] Tried to move back new op lower bound from 7317255343802068992 to 7317255343485448192. Current Snapshot: MvccSnapshot[applied={T|T < 7317255343802068992 or (T in {7317255343802068992})}]
I20260811 09:08:12.952194 4698 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43350:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:08:12.952517 4698 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43350:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:08:12.957517 4698 catalog_manager.cc:5782] T 00000000000000000000000000000000 P eeabb7f4c99e4c14813269d0e0bb1761: Sending DeleteTablet for 3 replicas of tablet e1eaac31db1f498c8731c682a8939b6a
I20260811 09:08:12.958971 4926 tablet_service.cc:1514] Processing DeleteTablet for tablet e1eaac31db1f498c8731c682a8939b6a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:08:12 UTC) from {username='kudu', principal='kudu/127.28.154.190@KRBTEST.COM'} at 127.0.0.1:35082
I20260811 09:08:12.958964 5159 tablet_service.cc:1514] Processing DeleteTablet for tablet e1eaac31db1f498c8731c682a8939b6a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:08:12 UTC) from {username='kudu', principal='kudu/127.28.154.190@KRBTEST.COM'} at 127.0.0.1:59220
I20260811 09:08:12.958978 5395 tablet_service.cc:1514] Processing DeleteTablet for tablet e1eaac31db1f498c8731c682a8939b6a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:08:12 UTC) from {username='kudu', principal='kudu/127.28.154.190@KRBTEST.COM'} at 127.0.0.1:37930
I20260811 09:08:12.961030 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 4811
I20260811 09:08:12.969978 5593 tablet_replica.cc:330] T e1eaac31db1f498c8731c682a8939b6a P 689189b7100d4fd4b05f2990370e97db: stopping tablet replica
I20260811 09:08:12.971375 5593 raft_consensus.cc:2229] T e1eaac31db1f498c8731c682a8939b6a P 689189b7100d4fd4b05f2990370e97db [term 1 LEADER]: Raft consensus shutting down.
I20260811 09:08:12.974066 5593 raft_consensus.cc:2258] T e1eaac31db1f498c8731c682a8939b6a P 689189b7100d4fd4b05f2990370e97db [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:08:12.979077 5593 ts_tablet_manager.cc:1905] T e1eaac31db1f498c8731c682a8939b6a P 689189b7100d4fd4b05f2990370e97db: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:08:12.995296 5594 tablet_replica.cc:330] T e1eaac31db1f498c8731c682a8939b6a P c8b06da9c2dd4a028e0038446f104cc8: stopping tablet replica
I20260811 09:08:12.996188 5593 ts_tablet_manager.cc:1918] T e1eaac31db1f498c8731c682a8939b6a P 689189b7100d4fd4b05f2990370e97db: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:08:12.996503 5593 log.cc:1196] T e1eaac31db1f498c8731c682a8939b6a P 689189b7100d4fd4b05f2990370e97db: Deleting WAL directory at /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439250192320-29290-0/minicluster-data/ts-0/wal/wals/e1eaac31db1f498c8731c682a8939b6a
I20260811 09:08:12.997224 5593 ts_tablet_manager.cc:1939] T e1eaac31db1f498c8731c682a8939b6a P 689189b7100d4fd4b05f2990370e97db: Deleting consensus metadata
I20260811 09:08:12.999128 5594 raft_consensus.cc:2229] T e1eaac31db1f498c8731c682a8939b6a P c8b06da9c2dd4a028e0038446f104cc8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:08:12.999339 4681 catalog_manager.cc:4841] TS 689189b7100d4fd4b05f2990370e97db (127.28.154.129:44273): tablet e1eaac31db1f498c8731c682a8939b6a (table test-table [id=c5fb6454b4ca415689977d5f57a1c969]) successfully deleted
I20260811 09:08:12.999768 5594 raft_consensus.cc:2258] T e1eaac31db1f498c8731c682a8939b6a P c8b06da9c2dd4a028e0038446f104cc8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:08:13.002419 5594 ts_tablet_manager.cc:1905] T e1eaac31db1f498c8731c682a8939b6a P c8b06da9c2dd4a028e0038446f104cc8: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 11 09:08:13 dist-test-slave-xt4k krb5kdc[4618](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439290, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.154.129@KRBTEST.COM
I20260811 09:08:13.008816 5596 tablet_replica.cc:330] T e1eaac31db1f498c8731c682a8939b6a P 1343406c45a64eb4b66f49e58bd371fd: stopping tablet replica
I20260811 09:08:13.009415 5596 raft_consensus.cc:2229] T e1eaac31db1f498c8731c682a8939b6a P 1343406c45a64eb4b66f49e58bd371fd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:08:13.010159 5594 ts_tablet_manager.cc:1918] T e1eaac31db1f498c8731c682a8939b6a P c8b06da9c2dd4a028e0038446f104cc8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:08:13.010411 5594 log.cc:1196] T e1eaac31db1f498c8731c682a8939b6a P c8b06da9c2dd4a028e0038446f104cc8: Deleting WAL directory at /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439250192320-29290-0/minicluster-data/ts-1/wal/wals/e1eaac31db1f498c8731c682a8939b6a
I20260811 09:08:13.011020 5594 ts_tablet_manager.cc:1939] T e1eaac31db1f498c8731c682a8939b6a P c8b06da9c2dd4a028e0038446f104cc8: Deleting consensus metadata
I20260811 09:08:13.011442 5596 raft_consensus.cc:2258] T e1eaac31db1f498c8731c682a8939b6a P 1343406c45a64eb4b66f49e58bd371fd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:08:13.013072 4675 catalog_manager.cc:4841] TS c8b06da9c2dd4a028e0038446f104cc8 (127.28.154.130:34479): tablet e1eaac31db1f498c8731c682a8939b6a (table test-table [id=c5fb6454b4ca415689977d5f57a1c969]) successfully deleted
I20260811 09:08:13.014356 5596 ts_tablet_manager.cc:1905] T e1eaac31db1f498c8731c682a8939b6a P 1343406c45a64eb4b66f49e58bd371fd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:08:13.018653 5542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:12.962301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35098 (local address 127.28.154.129:44273)
0811 09:08:12.962525 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:12.962541 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:12.988980 (+ 26439us) server_negotiation.cc:407] Connection header received
0811 09:08:12.989369 (+ 389us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:12.989376 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:12.989485 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:12.989594 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:12.996013 (+ 6419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:12.997302 (+ 1289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:13.000172 (+ 2870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:13.001036 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:13.001160 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:13.009922 (+ 8762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:13.009943 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:13.009963 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:08:13.010021 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:13.014540 (+ 4519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:13.015656 (+ 1116us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:13.015667 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:13.015671 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:13.015878 (+ 207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:13.016457 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:13.016463 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:13.016466 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:13.016934 (+ 468us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:13.017236 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:13.017906 (+ 670us) server_negotiation.cc:299] Negotiation successful
0811 09:08:13.018247 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260811 09:08:13.020442 5023 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:08:13.022862 5596 ts_tablet_manager.cc:1918] T e1eaac31db1f498c8731c682a8939b6a P 1343406c45a64eb4b66f49e58bd371fd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:08:13.023062 5596 log.cc:1196] T e1eaac31db1f498c8731c682a8939b6a P 1343406c45a64eb4b66f49e58bd371fd: Deleting WAL directory at /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786439250192320-29290-0/minicluster-data/ts-2/wal/wals/e1eaac31db1f498c8731c682a8939b6a
I20260811 09:08:13.023617 5596 ts_tablet_manager.cc:1939] T e1eaac31db1f498c8731c682a8939b6a P 1343406c45a64eb4b66f49e58bd371fd: Deleting consensus metadata
I20260811 09:08:13.025699 4681 catalog_manager.cc:4841] TS 1343406c45a64eb4b66f49e58bd371fd (127.28.154.131:35779): tablet e1eaac31db1f498c8731c682a8939b6a (table test-table [id=c5fb6454b4ca415689977d5f57a1c969]) successfully deleted
I20260811 09:08:13.124539 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 4811
I20260811 09:08:13.152526 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 5039
Aug 11 09:08:13 dist-test-slave-xt4k krb5kdc[4618](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439290, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.154.130@KRBTEST.COM
I20260811 09:08:13.168931 5532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:13.153355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59224 (local address 127.28.154.130:34479)
0811 09:08:13.153536 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:13.153541 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:13.153559 (+ 18us) server_negotiation.cc:407] Connection header received
0811 09:08:13.153667 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:13.153671 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:13.153738 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:13.153830 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:13.155289 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:13.156160 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:13.157663 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:13.158188 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:13.158315 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:13.162781 (+ 4466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:13.162799 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:13.162819 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:08:13.162869 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:13.165961 (+ 3092us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:13.166721 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:13.166729 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:13.166732 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:13.166882 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:13.167293 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:13.167298 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:13.167300 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:13.167801 (+ 501us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:13.168056 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:13.168325 (+ 269us) server_negotiation.cc:299] Negotiation successful
0811 09:08:13.168593 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 09:08:13.169740 5269 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:08:13.260049 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 5039
I20260811 09:08:13.288148 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 5290
Aug 11 09:08:13 dist-test-slave-xt4k krb5kdc[4618](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439290, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.154.131@KRBTEST.COM
I20260811 09:08:13.306177 5539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:13.289119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37944 (local address 127.28.154.131:35779)
0811 09:08:13.289475 (+ 356us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:13.289482 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:13.289503 (+ 21us) server_negotiation.cc:407] Connection header received
0811 09:08:13.289615 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:13.289627 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:13.289722 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:13.289833 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:13.291499 (+ 1666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:13.292315 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:13.294026 (+ 1711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:13.294496 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:13.294628 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:13.299778 (+ 5150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:13.299796 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:13.299815 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:08:13.299861 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:13.302958 (+ 3097us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:13.303738 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:13.303747 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:13.303750 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:13.303917 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:13.304333 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:13.304364 (+ 31us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:13.304368 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:13.304885 (+ 517us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:13.305221 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:13.305597 (+ 376us) server_negotiation.cc:299] Negotiation successful
0811 09:08:13.305834 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":186}
I20260811 09:08:13.306885 5504 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:08:13.397058 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 5290
I20260811 09:08:13.416376 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 4642
I20260811 09:08:13.417572 4752 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:08:13.528630 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 4642
2026-08-11T09:08:13Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4673 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-11T09:08:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:08:13Z Disabled control of system clock
I20260811 09:08:13.611673 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439250192320-29290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439250192320-29290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439250192320-29290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.154.190:35113
--webserver_interface=127.28.154.190
--webserver_port=0
--builtin_ntp_servers=127.28.154.148:44883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.154.190:35113
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 09:08:13.969022 5609 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:08:13.969350 5609 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:08:13.969445 5609 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:08:13.977228 5609 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:08:13.977381 5609 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:08:13.977423 5609 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:08:13.977483 5609 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:08:13.977528 5609 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:08:13.987838 5609 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:44883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439250192320-29290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439250192320-29290-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.154.190:35113
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.154.190:35113
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.154.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439250192320-29290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:08:13.989243 5609 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:08:13.991080 5609 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:08:14.003975 5617 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:08:14.003998 5616 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:08:14.004413 5609 server_base.cc:1034] running on GCE node
W20260811 09:08:14.003975 5620 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:08:14.005602 5609 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:08:14.007109 5609 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:08:14.008473 5609 hybrid_clock.cc:648] HybridClock initialized: now 1786439294008279 us; error 65 us; skew 500 ppm
I20260811 09:08:14.008939 5609 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 09:08:14.014825 5609 webserver.cc:460] Webserver started at http://127.28.154.190:43193/ using document root <none> and password file <none>
I20260811 09:08:14.015683 5609 fs_manager.cc:359] Metadata directory not provided
I20260811 09:08:14.015796 5609 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:08:14.016094 5609 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:08:14.019352 5609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439250192320-29290-0/minicluster-data/master-0/data/instance:
uuid: "274f21b7412140d0a093422fdd96c125"
format_stamp: "Formatted at 2026-08-11 09:08:14 on dist-test-slave-xt4k"
server_key: "cbd47d453eb5cc69f00b1076cd78c33f"
server_key_iv: "bacae1dc7054714605ea27f5179d085d"
server_key_version: "encryptionkey@0"
I20260811 09:08:14.020198 5609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439250192320-29290-0/minicluster-data/master-0/wal/instance:
uuid: "274f21b7412140d0a093422fdd96c125"
format_stamp: "Formatted at 2026-08-11 09:08:14 on dist-test-slave-xt4k"
server_key: "cbd47d453eb5cc69f00b1076cd78c33f"
server_key_iv: "bacae1dc7054714605ea27f5179d085d"
server_key_version: "encryptionkey@0"
I20260811 09:08:14.026573 5609 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260811 09:08:14.030659 5626 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:08:14.032318 5609 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 09:08:14.032579 5609 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439250192320-29290-0/minicluster-data/master-0/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439250192320-29290-0/minicluster-data/master-0/wal
uuid: "274f21b7412140d0a093422fdd96c125"
format_stamp: "Formatted at 2026-08-11 09:08:14 on dist-test-slave-xt4k"
server_key: "cbd47d453eb5cc69f00b1076cd78c33f"
server_key_iv: "bacae1dc7054714605ea27f5179d085d"
server_key_version: "encryptionkey@0"
I20260811 09:08:14.032806 5609 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439250192320-29290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439250192320-29290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439250192320-29290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:08:14.039306 5609 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:08:14.040170 5609 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:08:14.040542 5609 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:08:14.060251 5609 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.190:35113
I20260811 09:08:14.060256 5724 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.190:35113 every 8 connection(s)
I20260811 09:08:14.061969 5609 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
I20260811 09:08:14.066254 5725 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:08:14.066496 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 5609
I20260811 09:08:14.066687 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439250192320-29290-0/minicluster-data/master-0/wal/instance
I20260811 09:08:14.067310 29290 external_mini_cluster.cc:1428] Setting key e1fe576f149fe643da213a5ce752e915
I20260811 09:08:14.076577 5730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:14.069636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47220 (local address 127.28.154.190:35113)
0811 09:08:14.070792 (+ 1156us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:14.070807 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:14.070853 (+ 46us) server_negotiation.cc:407] Connection header received
0811 09:08:14.072610 (+ 1757us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:14.072640 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:14.073147 (+ 507us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:14.073534 (+ 387us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:14.074772 (+ 1238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:14.074780 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:14.074795 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:08:14.074812 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:14.074888 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:08:14.074906 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:14.075090 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:14.075219 (+ 129us) server_negotiation.cc:299] Negotiation successful
0811 09:08:14.075523 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":659,"thread_start_us":405,"threads_started":1}
I20260811 09:08:14.080479 5725 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 274f21b7412140d0a093422fdd96c125: Bootstrap starting.
I20260811 09:08:14.083832 5725 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 274f21b7412140d0a093422fdd96c125: Neither blocks nor log segments found. Creating new log.
I20260811 09:08:14.084949 5725 log.cc:824] T 00000000000000000000000000000000 P 274f21b7412140d0a093422fdd96c125: Log is configured to *not* fsync() on all Append() calls
I20260811 09:08:14.088161 5725 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 274f21b7412140d0a093422fdd96c125: No bootstrap required, opened a new log
I20260811 09:08:14.097267 5725 raft_consensus.cc:348] T 00000000000000000000000000000000 P 274f21b7412140d0a093422fdd96c125 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "274f21b7412140d0a093422fdd96c125" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 35113 } }
I20260811 09:08:14.097792 5725 raft_consensus.cc:374] T 00000000000000000000000000000000 P 274f21b7412140d0a093422fdd96c125 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:08:14.097965 5725 raft_consensus.cc:729] T 00000000000000000000000000000000 P 274f21b7412140d0a093422fdd96c125 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 274f21b7412140d0a093422fdd96c125, State: Initialized, Role: FOLLOWER
I20260811 09:08:14.098784 5725 consensus_queue.cc:260] T 00000000000000000000000000000000 P 274f21b7412140d0a093422fdd96c125 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "274f21b7412140d0a093422fdd96c125" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 35113 } }
I20260811 09:08:14.099026 5725 raft_consensus.cc:388] T 00000000000000000000000000000000 P 274f21b7412140d0a093422fdd96c125 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:08:14.099148 5725 raft_consensus.cc:482] T 00000000000000000000000000000000 P 274f21b7412140d0a093422fdd96c125 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:08:14.099289 5725 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 274f21b7412140d0a093422fdd96c125 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:08:14.101600 5725 raft_consensus.cc:504] T 00000000000000000000000000000000 P 274f21b7412140d0a093422fdd96c125 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "274f21b7412140d0a093422fdd96c125" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 35113 } }
I20260811 09:08:14.102180 5725 leader_election.cc:302] T 00000000000000000000000000000000 P 274f21b7412140d0a093422fdd96c125 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 274f21b7412140d0a093422fdd96c125; no voters:
I20260811 09:08:14.102895 5725 leader_election.cc:288] T 00000000000000000000000000000000 P 274f21b7412140d0a093422fdd96c125 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:08:14.103078 5735 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 274f21b7412140d0a093422fdd96c125 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:08:14.103834 5735 raft_consensus.cc:686] T 00000000000000000000000000000000 P 274f21b7412140d0a093422fdd96c125 [term 1 LEADER]: Becoming Leader. State: Replica: 274f21b7412140d0a093422fdd96c125, State: Running, Role: LEADER
I20260811 09:08:14.104580 5735 consensus_queue.cc:237] T 00000000000000000000000000000000 P 274f21b7412140d0a093422fdd96c125 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "274f21b7412140d0a093422fdd96c125" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 35113 } }
I20260811 09:08:14.105168 5725 sys_catalog.cc:564] T 00000000000000000000000000000000 P 274f21b7412140d0a093422fdd96c125 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:08:14.108400 5736 sys_catalog.cc:455] T 00000000000000000000000000000000 P 274f21b7412140d0a093422fdd96c125 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "274f21b7412140d0a093422fdd96c125" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "274f21b7412140d0a093422fdd96c125" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 35113 } } }
I20260811 09:08:14.108650 5737 sys_catalog.cc:455] T 00000000000000000000000000000000 P 274f21b7412140d0a093422fdd96c125 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 274f21b7412140d0a093422fdd96c125. Latest consensus state: current_term: 1 leader_uuid: "274f21b7412140d0a093422fdd96c125" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "274f21b7412140d0a093422fdd96c125" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 35113 } } }
I20260811 09:08:14.109100 5736 sys_catalog.cc:458] T 00000000000000000000000000000000 P 274f21b7412140d0a093422fdd96c125 [sys.catalog]: This master's current role is: LEADER
I20260811 09:08:14.109184 5737 sys_catalog.cc:458] T 00000000000000000000000000000000 P 274f21b7412140d0a093422fdd96c125 [sys.catalog]: This master's current role is: LEADER
I20260811 09:08:14.110056 5741 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:08:14.116643 5741 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:08:14.127231 5741 catalog_manager.cc:1347] Generated new cluster ID: 670a8ec16e954b5f844b1c3fbdc75449
I20260811 09:08:14.127378 5741 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:08:14.156041 5741 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:08:14.156531 5741 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:08:14.165958 5741 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 274f21b7412140d0a093422fdd96c125: Generated new TSK 0
I20260811 09:08:14.167217 5741 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 09:08:14.181424 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439250192320-29290-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439250192320-29290-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439250192320-29290-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439250192320-29290-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.154.129:0
--local_ip_for_outbound_sockets=127.28.154.129
--webserver_interface=127.28.154.129
--webserver_port=0
--tserver_master_addrs=127.28.154.190:35113
--builtin_ntp_servers=127.28.154.148:44883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 09:08:14.532814 5766 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:08:14.533124 5766 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:08:14.533236 5766 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:08:14.540669 5766 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:08:14.540786 5766 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:08:14.540897 5766 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.154.129
I20260811 09:08:14.550365 5766 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:44883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439250192320-29290-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439250192320-29290-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.154.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439250192320-29290-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.154.129
--webserver_port=0
--tserver_master_addrs=127.28.154.190:35113
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.154.129
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439250192320-29290-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:08:14.551667 5766 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:08:14.553493 5766 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:08:14.567196 5780 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:08:14.567242 5778 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:08:14.567163 5777 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:08:14.567880 5766 server_base.cc:1034] running on GCE node
I20260811 09:08:14.568634 5766 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:08:14.570123 5766 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:08:14.571353 5766 hybrid_clock.cc:648] HybridClock initialized: now 1786439294571288 us; error 58 us; skew 500 ppm
I20260811 09:08:14.571712 5766 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 09:08:14.574831 5766 webserver.cc:460] Webserver started at http://127.28.154.129:33995/ using document root <none> and password file <none>
I20260811 09:08:14.575647 5766 fs_manager.cc:359] Metadata directory not provided
I20260811 09:08:14.575753 5766 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:08:14.576076 5766 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:08:14.579546 5766 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439250192320-29290-0/minicluster-data/ts-0/data/instance:
uuid: "a0af6088cbad49b095e66ad2031d3b2a"
format_stamp: "Formatted at 2026-08-11 09:08:14 on dist-test-slave-xt4k"
server_key: "c5efb938aca4e8800f11a42f280d63f9"
server_key_iv: "5f7c058120c77aa3c3af5495df2106e2"
server_key_version: "encryptionkey@0"
I20260811 09:08:14.580741 5766 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439250192320-29290-0/minicluster-data/ts-0/wal/instance:
uuid: "a0af6088cbad49b095e66ad2031d3b2a"
format_stamp: "Formatted at 2026-08-11 09:08:14 on dist-test-slave-xt4k"
server_key: "c5efb938aca4e8800f11a42f280d63f9"
server_key_iv: "5f7c058120c77aa3c3af5495df2106e2"
server_key_version: "encryptionkey@0"
I20260811 09:08:14.586838 5766 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260811 09:08:14.590824 5793 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:08:14.592300 5766 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260811 09:08:14.592521 5766 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439250192320-29290-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439250192320-29290-0/minicluster-data/ts-0/wal
uuid: "a0af6088cbad49b095e66ad2031d3b2a"
format_stamp: "Formatted at 2026-08-11 09:08:14 on dist-test-slave-xt4k"
server_key: "c5efb938aca4e8800f11a42f280d63f9"
server_key_iv: "5f7c058120c77aa3c3af5495df2106e2"
server_key_version: "encryptionkey@0"
I20260811 09:08:14.592769 5766 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439250192320-29290-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439250192320-29290-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439250192320-29290-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:08:14.598774 5766 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:08:14.599905 5766 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:08:14.600232 5766 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:08:14.601737 5766 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:08:14.603399 5766 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:08:14.603492 5766 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:08:14.603580 5766 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:08:14.603633 5766 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:08:14.638383 5766 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.129:37001
I20260811 09:08:14.638463 5994 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.129:37001 every 8 connection(s)
I20260811 09:08:14.639765 5766 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439250192320-29290-0/minicluster-data/ts-0/data/info.pb
I20260811 09:08:14.645884 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 5766
I20260811 09:08:14.646111 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439250192320-29290-0/minicluster-data/ts-0/wal/instance
I20260811 09:08:14.646896 29290 external_mini_cluster.cc:1428] Setting key efc59312868ec2aa253b8e05022749d3
I20260811 09:08:14.648703 5999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:14.642519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.129:43401 (local address 127.28.154.190:35113)
0811 09:08:14.642975 (+ 456us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:14.642980 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:14.644126 (+ 1146us) server_negotiation.cc:407] Connection header received
0811 09:08:14.645919 (+ 1793us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:14.645925 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:14.645998 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:14.646112 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:14.647834 (+ 1722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:14.647841 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:14.647846 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:08:14.647878 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:14.647936 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:08:14.647954 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:14.648072 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:14.648277 (+ 205us) server_negotiation.cc:299] Negotiation successful
0811 09:08:14.648369 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":246,"threads_started":1}
I20260811 09:08:14.649713 6001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:14.642978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:35113 (local address 127.28.154.129:43401)
0811 09:08:14.643947 (+ 969us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:14.643990 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:14.645621 (+ 1631us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:14.646440 (+ 819us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:14.646465 (+ 25us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:14.646599 (+ 134us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:08:14.647416 (+ 817us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 09:08:14.647420 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:08:14.647488 (+ 68us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 09:08:14.647615 (+ 127us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:08:14.648058 (+ 443us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:08:14.648068 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:08:14.648076 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 09:08:14.648329 (+ 253us) client_negotiation.cc:240] Negotiation successful
0811 09:08:14.648623 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":271,"threads_started":1}
I20260811 09:08:14.650254 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439250192320-29290-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439250192320-29290-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439250192320-29290-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439250192320-29290-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.154.130:0
--local_ip_for_outbound_sockets=127.28.154.130
--webserver_interface=127.28.154.130
--webserver_port=0
--tserver_master_addrs=127.28.154.190:35113
--builtin_ntp_servers=127.28.154.148:44883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260811 09:08:14.651240 5997 heartbeater.cc:347] Connected to a master server at 127.28.154.190:35113
I20260811 09:08:14.651561 5997 heartbeater.cc:464] Registering TS with master...
I20260811 09:08:14.652155 5997 heartbeater.cc:511] Master 127.28.154.190:35113 requested a full tablet report, sending...
I20260811 09:08:14.654229 5659 ts_manager.cc:194] Registered new tserver with Master: a0af6088cbad49b095e66ad2031d3b2a (127.28.154.129:37001)
W20260811 09:08:15.005838 6002 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:08:15.006155 6002 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:08:15.006260 6002 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:08:15.013690 6002 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:08:15.013796 6002 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:08:15.013908 6002 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.154.130
I20260811 09:08:15.023270 6002 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:44883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439250192320-29290-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439250192320-29290-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.154.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439250192320-29290-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.154.130
--webserver_port=0
--tserver_master_addrs=127.28.154.190:35113
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.154.130
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439250192320-29290-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:08:15.024698 6002 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:08:15.026429 6002 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:08:15.039588 6018 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:08:15.039520 6021 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:08:15.039564 6019 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:08:15.040421 6002 server_base.cc:1034] running on GCE node
I20260811 09:08:15.041059 6002 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:08:15.042014 6002 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:08:15.043259 6002 hybrid_clock.cc:648] HybridClock initialized: now 1786439295043196 us; error 53 us; skew 500 ppm
I20260811 09:08:15.043588 6002 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 09:08:15.045861 6002 webserver.cc:460] Webserver started at http://127.28.154.130:39785/ using document root <none> and password file <none>
I20260811 09:08:15.046564 6002 fs_manager.cc:359] Metadata directory not provided
I20260811 09:08:15.046669 6002 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:08:15.046990 6002 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:08:15.049916 6002 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439250192320-29290-0/minicluster-data/ts-1/data/instance:
uuid: "8388295b42914507b22f8a4099e79a02"
format_stamp: "Formatted at 2026-08-11 09:08:15 on dist-test-slave-xt4k"
server_key: "8545a54df682bef0c6b008f34d306cb7"
server_key_iv: "6daf702a472b5cfc8cc1c5e7cdb425ea"
server_key_version: "encryptionkey@0"
I20260811 09:08:15.050729 6002 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439250192320-29290-0/minicluster-data/ts-1/wal/instance:
uuid: "8388295b42914507b22f8a4099e79a02"
format_stamp: "Formatted at 2026-08-11 09:08:15 on dist-test-slave-xt4k"
server_key: "8545a54df682bef0c6b008f34d306cb7"
server_key_iv: "6daf702a472b5cfc8cc1c5e7cdb425ea"
server_key_version: "encryptionkey@0"
I20260811 09:08:15.056061 6002 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 09:08:15.059803 6032 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:08:15.061462 6002 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 09:08:15.061679 6002 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439250192320-29290-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439250192320-29290-0/minicluster-data/ts-1/wal
uuid: "8388295b42914507b22f8a4099e79a02"
format_stamp: "Formatted at 2026-08-11 09:08:15 on dist-test-slave-xt4k"
server_key: "8545a54df682bef0c6b008f34d306cb7"
server_key_iv: "6daf702a472b5cfc8cc1c5e7cdb425ea"
server_key_version: "encryptionkey@0"
I20260811 09:08:15.061926 6002 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439250192320-29290-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439250192320-29290-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439250192320-29290-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:08:15.069669 6002 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:08:15.070670 6002 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:08:15.070942 6002 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:08:15.072055 6002 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:08:15.073868 6002 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:08:15.073973 6002 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:08:15.074075 6002 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:08:15.074146 6002 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:08:15.108120 6002 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.130:42381
I20260811 09:08:15.108173 6255 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.130:42381 every 8 connection(s)
I20260811 09:08:15.109488 6002 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439250192320-29290-0/minicluster-data/ts-1/data/info.pb
I20260811 09:08:15.115412 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 6002
I20260811 09:08:15.115718 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439250192320-29290-0/minicluster-data/ts-1/wal/instance
I20260811 09:08:15.116294 29290 external_mini_cluster.cc:1428] Setting key af6f8f67dca894daec9a22d9671a469d
I20260811 09:08:15.119350 5999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:15.112451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.130:55223 (local address 127.28.154.190:35113)
0811 09:08:15.112651 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:15.112656 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:15.114230 (+ 1574us) server_negotiation.cc:407] Connection header received
0811 09:08:15.116792 (+ 2562us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:15.116797 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:15.116878 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:15.116989 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:15.118508 (+ 1519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:15.118514 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:15.118518 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:08:15.118539 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:15.118574 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:08:15.118587 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:15.118764 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:15.118987 (+ 223us) server_negotiation.cc:299] Negotiation successful
0811 09:08:15.119061 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 09:08:15.120132 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439250192320-29290-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439250192320-29290-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439250192320-29290-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439250192320-29290-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.154.131:0
--local_ip_for_outbound_sockets=127.28.154.131
--webserver_interface=127.28.154.131
--webserver_port=0
--tserver_master_addrs=127.28.154.190:35113
--builtin_ntp_servers=127.28.154.148:44883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260811 09:08:15.120559 6260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:15.112949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:35113 (local address 127.28.154.130:55223)
0811 09:08:15.114054 (+ 1105us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:15.114100 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:15.116383 (+ 2283us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:15.117270 (+ 887us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:15.117284 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:15.117438 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:08:15.118159 (+ 721us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 09:08:15.118162 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:08:15.118205 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 09:08:15.118307 (+ 102us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:08:15.118732 (+ 425us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:08:15.118751 (+ 19us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:08:15.118761 (+ 10us) client_negotiation.cc:769] Sending connection context
0811 09:08:15.119087 (+ 326us) client_negotiation.cc:240] Negotiation successful
0811 09:08:15.119398 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":582,"thread_start_us":352,"threads_started":1}
I20260811 09:08:15.122227 6256 heartbeater.cc:347] Connected to a master server at 127.28.154.190:35113
I20260811 09:08:15.122609 6256 heartbeater.cc:464] Registering TS with master...
I20260811 09:08:15.123276 6256 heartbeater.cc:511] Master 127.28.154.190:35113 requested a full tablet report, sending...
I20260811 09:08:15.125010 5659 ts_manager.cc:194] Registered new tserver with Master: 8388295b42914507b22f8a4099e79a02 (127.28.154.130:42381)
W20260811 09:08:15.455681 6263 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:08:15.456002 6263 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:08:15.456104 6263 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:08:15.463663 6263 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:08:15.463785 6263 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:08:15.463897 6263 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.154.131
I20260811 09:08:15.473649 6263 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:44883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439250192320-29290-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439250192320-29290-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.154.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439250192320-29290-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.154.131
--webserver_port=0
--tserver_master_addrs=127.28.154.190:35113
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.154.131
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439250192320-29290-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:08:15.474942 6263 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:08:15.476830 6263 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:08:15.491158 6276 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:08:15.491212 6279 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:08:15.491189 6272 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:08:15.492754 6263 server_base.cc:1034] running on GCE node
I20260811 09:08:15.493515 6263 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:08:15.494535 6263 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:08:15.495769 6263 hybrid_clock.cc:648] HybridClock initialized: now 1786439295495698 us; error 58 us; skew 500 ppm
I20260811 09:08:15.496111 6263 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 09:08:15.498574 6263 webserver.cc:460] Webserver started at http://127.28.154.131:43431/ using document root <none> and password file <none>
I20260811 09:08:15.499310 6263 fs_manager.cc:359] Metadata directory not provided
I20260811 09:08:15.499425 6263 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:08:15.499758 6263 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:08:15.502785 6263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439250192320-29290-0/minicluster-data/ts-2/data/instance:
uuid: "ab68d29fc55448dd826c71661e30a472"
format_stamp: "Formatted at 2026-08-11 09:08:15 on dist-test-slave-xt4k"
server_key: "daa89fa25e38cf205d85a7f6b1788193"
server_key_iv: "bb1d65ad9febd13b382a0a2383f74567"
server_key_version: "encryptionkey@0"
I20260811 09:08:15.503659 6263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439250192320-29290-0/minicluster-data/ts-2/wal/instance:
uuid: "ab68d29fc55448dd826c71661e30a472"
format_stamp: "Formatted at 2026-08-11 09:08:15 on dist-test-slave-xt4k"
server_key: "daa89fa25e38cf205d85a7f6b1788193"
server_key_iv: "bb1d65ad9febd13b382a0a2383f74567"
server_key_version: "encryptionkey@0"
I20260811 09:08:15.509307 6263 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.003s
I20260811 09:08:15.513120 6285 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:08:15.514847 6263 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 09:08:15.515007 6263 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439250192320-29290-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439250192320-29290-0/minicluster-data/ts-2/wal
uuid: "ab68d29fc55448dd826c71661e30a472"
format_stamp: "Formatted at 2026-08-11 09:08:15 on dist-test-slave-xt4k"
server_key: "daa89fa25e38cf205d85a7f6b1788193"
server_key_iv: "bb1d65ad9febd13b382a0a2383f74567"
server_key_version: "encryptionkey@0"
I20260811 09:08:15.515218 6263 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439250192320-29290-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439250192320-29290-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439250192320-29290-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:08:15.520998 6263 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:08:15.521775 6263 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:08:15.522011 6263 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:08:15.522941 6263 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:08:15.524623 6263 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:08:15.524727 6263 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:08:15.524812 6263 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:08:15.524866 6263 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:08:15.554807 6263 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.131:45573
I20260811 09:08:15.554884 6502 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.131:45573 every 8 connection(s)
I20260811 09:08:15.556130 6263 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439250192320-29290-0/minicluster-data/ts-2/data/info.pb
I20260811 09:08:15.564325 5999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:15.558811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.131:53379 (local address 127.28.154.190:35113)
0811 09:08:15.559017 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:15.559021 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:15.560396 (+ 1375us) server_negotiation.cc:407] Connection header received
0811 09:08:15.562078 (+ 1682us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:15.562083 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:15.562141 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:15.562236 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:15.563688 (+ 1452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:15.563694 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:15.563697 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:08:15.563717 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:15.563753 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:08:15.563765 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:15.563872 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:15.564043 (+ 171us) server_negotiation.cc:299] Negotiation successful
0811 09:08:15.564098 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 09:08:15.564950 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 6263
I20260811 09:08:15.565158 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439250192320-29290-0/minicluster-data/ts-2/wal/instance
I20260811 09:08:15.565711 29290 external_mini_cluster.cc:1428] Setting key f082b5887412e50a77af8ddc9b52abb9
I20260811 09:08:15.565730 6509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:15.559310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:35113 (local address 127.28.154.131:53379)
0811 09:08:15.560238 (+ 928us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:15.560279 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:15.561822 (+ 1543us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:15.562486 (+ 664us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:15.562494 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:15.562617 (+ 123us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:08:15.563363 (+ 746us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 09:08:15.563366 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:08:15.563407 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 09:08:15.563502 (+ 95us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:08:15.563869 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:08:15.563878 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:08:15.563884 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 09:08:15.564112 (+ 228us) client_negotiation.cc:240] Negotiation successful
0811 09:08:15.564318 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":308,"threads_started":1}
I20260811 09:08:15.567164 6505 heartbeater.cc:347] Connected to a master server at 127.28.154.190:35113
I20260811 09:08:15.567473 6505 heartbeater.cc:464] Registering TS with master...
I20260811 09:08:15.568084 6505 heartbeater.cc:511] Master 127.28.154.190:35113 requested a full tablet report, sending...
I20260811 09:08:15.569499 5658 ts_manager.cc:194] Registered new tserver with Master: ab68d29fc55448dd826c71661e30a472 (127.28.154.131:45573)
I20260811 09:08:15.581277 29290 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 09:08:15.588330 5999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:15.585918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44026 (local address 127.28.154.190:35113)
0811 09:08:15.586101 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:15.586105 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:15.586618 (+ 513us) server_negotiation.cc:407] Connection header received
0811 09:08:15.586761 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:15.586764 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:15.586821 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:15.586911 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:15.587822 (+ 911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:15.587827 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:15.587831 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:08:15.587848 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:15.587886 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:08:15.587899 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:15.588026 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:15.588063 (+ 37us) server_negotiation.cc:299] Negotiation successful
0811 09:08:15.588113 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 09:08:15.592758 5658 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44026:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 09:08:15.599943 5658 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 09:08:15.620368 6530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:15.612798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.130:42381 (local address 127.0.0.1:55114)
0811 09:08:15.615692 (+ 2894us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:15.615732 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:15.615901 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:15.617085 (+ 1184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:15.617094 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:15.617195 (+ 101us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:08:15.618216 (+ 1021us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 09:08:15.618220 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:08:15.618253 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 09:08:15.618356 (+ 103us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:08:15.618794 (+ 438us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:08:15.618806 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:08:15.618814 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 09:08:15.618986 (+ 172us) client_negotiation.cc:240] Negotiation successful
0811 09:08:15.619092 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2649,"thread_start_us":1821,"threads_started":1}
I20260811 09:08:15.621318 6529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:15.612866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35710 (local address 127.28.154.129:37001)
0811 09:08:15.614576 (+ 1710us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:15.614587 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:15.618732 (+ 4145us) server_negotiation.cc:407] Connection header received
0811 09:08:15.618828 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:15.618837 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:15.619054 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:15.619201 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:15.620138 (+ 937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:15.620145 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:15.620158 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:08:15.620176 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:15.620235 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:08:15.620263 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:15.620413 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:15.620891 (+ 478us) server_negotiation.cc:299] Negotiation successful
0811 09:08:15.621009 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1519,"thread_start_us":1384,"threads_started":1}
I20260811 09:08:15.621364 6534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:15.613488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55114 (local address 127.28.154.130:42381)
0811 09:08:15.615029 (+ 1541us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:15.615035 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:15.615873 (+ 838us) server_negotiation.cc:407] Connection header received
0811 09:08:15.616085 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:15.616094 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:15.616394 (+ 300us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:15.616550 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:15.618554 (+ 2004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:15.618563 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:15.618574 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:08:15.618598 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:15.618646 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:08:15.618663 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:15.620678 (+ 2015us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:15.620965 (+ 287us) server_negotiation.cc:299] Negotiation successful
0811 09:08:15.621061 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1391,"thread_start_us":1278,"threads_started":1}
I20260811 09:08:15.622151 6531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:15.612814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.129:37001 (local address 127.0.0.1:35710)
0811 09:08:15.618605 (+ 5791us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:15.618621 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:15.618725 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:15.619214 (+ 489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:15.619219 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:15.619248 (+ 29us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:08:15.619922 (+ 674us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 09:08:15.619925 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:08:15.619942 (+ 17us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 09:08:15.619986 (+ 44us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:08:15.620720 (+ 734us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:08:15.620726 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:08:15.620731 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 09:08:15.621646 (+ 915us) client_negotiation.cc:240] Negotiation successful
0811 09:08:15.621704 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5628,"thread_start_us":5478,"threads_started":1}
I20260811 09:08:15.622447 6538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:15.614620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54466 (local address 127.28.154.131:45573)
0811 09:08:15.615538 (+ 918us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:15.615546 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:15.618800 (+ 3254us) server_negotiation.cc:407] Connection header received
0811 09:08:15.619028 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:15.619034 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:15.619186 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:15.619313 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:15.621589 (+ 2276us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:15.621597 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:15.621609 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:08:15.621631 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:15.621695 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:08:15.621716 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:15.621886 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:15.621958 (+ 72us) server_negotiation.cc:299] Negotiation successful
0811 09:08:15.622059 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":753,"thread_start_us":587,"threads_started":1}
I20260811 09:08:15.622573 6539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:15.615291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.131:45573 (local address 127.0.0.1:54466)
0811 09:08:15.618701 (+ 3410us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:15.618715 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:15.618873 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:15.620377 (+ 1504us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:15.620381 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:15.620403 (+ 22us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:08:15.621339 (+ 936us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 09:08:15.621341 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:08:15.621354 (+ 13us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 09:08:15.621392 (+ 38us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:08:15.621832 (+ 440us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:08:15.621837 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:08:15.621842 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 09:08:15.622006 (+ 164us) client_negotiation.cc:240] Negotiation successful
0811 09:08:15.622064 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3309,"thread_start_us":2757,"threads_started":1}
I20260811 09:08:15.625494 6379 tablet_service.cc:1467] Processing CreateTablet for tablet 49e696e19146475c9c09235399297fb4 (DEFAULT_TABLE table=test-table [id=11d25002158d475994bb65ec6035dd39]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:08:15.625500 6118 tablet_service.cc:1467] Processing CreateTablet for tablet 49e696e19146475c9c09235399297fb4 (DEFAULT_TABLE table=test-table [id=11d25002158d475994bb65ec6035dd39]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:08:15.625592 5883 tablet_service.cc:1467] Processing CreateTablet for tablet 49e696e19146475c9c09235399297fb4 (DEFAULT_TABLE table=test-table [id=11d25002158d475994bb65ec6035dd39]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:08:15.626711 6379 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49e696e19146475c9c09235399297fb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:08:15.626858 6118 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49e696e19146475c9c09235399297fb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:08:15.626865 5883 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49e696e19146475c9c09235399297fb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:08:15.645493 6540 tablet_bootstrap.cc:492] T 49e696e19146475c9c09235399297fb4 P ab68d29fc55448dd826c71661e30a472: Bootstrap starting.
I20260811 09:08:15.647415 6544 tablet_bootstrap.cc:492] T 49e696e19146475c9c09235399297fb4 P a0af6088cbad49b095e66ad2031d3b2a: Bootstrap starting.
I20260811 09:08:15.649199 6543 tablet_bootstrap.cc:492] T 49e696e19146475c9c09235399297fb4 P 8388295b42914507b22f8a4099e79a02: Bootstrap starting.
I20260811 09:08:15.652503 6540 tablet_bootstrap.cc:654] T 49e696e19146475c9c09235399297fb4 P ab68d29fc55448dd826c71661e30a472: Neither blocks nor log segments found. Creating new log.
I20260811 09:08:15.653564 6543 tablet_bootstrap.cc:654] T 49e696e19146475c9c09235399297fb4 P 8388295b42914507b22f8a4099e79a02: Neither blocks nor log segments found. Creating new log.
I20260811 09:08:15.654695 6543 log.cc:824] T 49e696e19146475c9c09235399297fb4 P 8388295b42914507b22f8a4099e79a02: Log is configured to *not* fsync() on all Append() calls
I20260811 09:08:15.654790 6544 tablet_bootstrap.cc:654] T 49e696e19146475c9c09235399297fb4 P a0af6088cbad49b095e66ad2031d3b2a: Neither blocks nor log segments found. Creating new log.
I20260811 09:08:15.656035 6544 log.cc:824] T 49e696e19146475c9c09235399297fb4 P a0af6088cbad49b095e66ad2031d3b2a: Log is configured to *not* fsync() on all Append() calls
I20260811 09:08:15.656035 6540 log.cc:824] T 49e696e19146475c9c09235399297fb4 P ab68d29fc55448dd826c71661e30a472: Log is configured to *not* fsync() on all Append() calls
I20260811 09:08:15.657117 5997 heartbeater.cc:503] Master 127.28.154.190:35113 was elected leader, sending a full tablet report...
I20260811 09:08:15.657831 6543 tablet_bootstrap.cc:492] T 49e696e19146475c9c09235399297fb4 P 8388295b42914507b22f8a4099e79a02: No bootstrap required, opened a new log
I20260811 09:08:15.658138 6543 ts_tablet_manager.cc:1397] T 49e696e19146475c9c09235399297fb4 P 8388295b42914507b22f8a4099e79a02: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260811 09:08:15.659552 6544 tablet_bootstrap.cc:492] T 49e696e19146475c9c09235399297fb4 P a0af6088cbad49b095e66ad2031d3b2a: No bootstrap required, opened a new log
I20260811 09:08:15.659761 6540 tablet_bootstrap.cc:492] T 49e696e19146475c9c09235399297fb4 P ab68d29fc55448dd826c71661e30a472: No bootstrap required, opened a new log
I20260811 09:08:15.660002 6544 ts_tablet_manager.cc:1397] T 49e696e19146475c9c09235399297fb4 P a0af6088cbad49b095e66ad2031d3b2a: Time spent bootstrapping tablet: real 0.014s user 0.009s sys 0.000s
I20260811 09:08:15.660002 6540 ts_tablet_manager.cc:1397] T 49e696e19146475c9c09235399297fb4 P ab68d29fc55448dd826c71661e30a472: Time spent bootstrapping tablet: real 0.015s user 0.003s sys 0.003s
I20260811 09:08:15.666342 6543 raft_consensus.cc:348] T 49e696e19146475c9c09235399297fb4 P 8388295b42914507b22f8a4099e79a02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0af6088cbad49b095e66ad2031d3b2a" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 37001 } } peers { permanent_uuid: "8388295b42914507b22f8a4099e79a02" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 42381 } } peers { permanent_uuid: "ab68d29fc55448dd826c71661e30a472" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 45573 } }
I20260811 09:08:15.666786 6543 raft_consensus.cc:374] T 49e696e19146475c9c09235399297fb4 P 8388295b42914507b22f8a4099e79a02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:08:15.666882 6543 raft_consensus.cc:729] T 49e696e19146475c9c09235399297fb4 P 8388295b42914507b22f8a4099e79a02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8388295b42914507b22f8a4099e79a02, State: Initialized, Role: FOLLOWER
I20260811 09:08:15.667500 6543 consensus_queue.cc:260] T 49e696e19146475c9c09235399297fb4 P 8388295b42914507b22f8a4099e79a02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0af6088cbad49b095e66ad2031d3b2a" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 37001 } } peers { permanent_uuid: "8388295b42914507b22f8a4099e79a02" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 42381 } } peers { permanent_uuid: "ab68d29fc55448dd826c71661e30a472" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 45573 } }
I20260811 09:08:15.668692 6256 heartbeater.cc:503] Master 127.28.154.190:35113 was elected leader, sending a full tablet report...
I20260811 09:08:15.669176 6543 ts_tablet_manager.cc:1428] T 49e696e19146475c9c09235399297fb4 P 8388295b42914507b22f8a4099e79a02: Time spent starting tablet: real 0.011s user 0.012s sys 0.000s
I20260811 09:08:15.671568 6544 raft_consensus.cc:348] T 49e696e19146475c9c09235399297fb4 P a0af6088cbad49b095e66ad2031d3b2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0af6088cbad49b095e66ad2031d3b2a" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 37001 } } peers { permanent_uuid: "8388295b42914507b22f8a4099e79a02" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 42381 } } peers { permanent_uuid: "ab68d29fc55448dd826c71661e30a472" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 45573 } }
I20260811 09:08:15.671559 6540 raft_consensus.cc:348] T 49e696e19146475c9c09235399297fb4 P ab68d29fc55448dd826c71661e30a472 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0af6088cbad49b095e66ad2031d3b2a" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 37001 } } peers { permanent_uuid: "8388295b42914507b22f8a4099e79a02" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 42381 } } peers { permanent_uuid: "ab68d29fc55448dd826c71661e30a472" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 45573 } }
I20260811 09:08:15.672019 6544 raft_consensus.cc:374] T 49e696e19146475c9c09235399297fb4 P a0af6088cbad49b095e66ad2031d3b2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:08:15.672015 6540 raft_consensus.cc:374] T 49e696e19146475c9c09235399297fb4 P ab68d29fc55448dd826c71661e30a472 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:08:15.672124 6540 raft_consensus.cc:729] T 49e696e19146475c9c09235399297fb4 P ab68d29fc55448dd826c71661e30a472 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab68d29fc55448dd826c71661e30a472, State: Initialized, Role: FOLLOWER
I20260811 09:08:15.672117 6544 raft_consensus.cc:729] T 49e696e19146475c9c09235399297fb4 P a0af6088cbad49b095e66ad2031d3b2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0af6088cbad49b095e66ad2031d3b2a, State: Initialized, Role: FOLLOWER
I20260811 09:08:15.672889 6540 consensus_queue.cc:260] T 49e696e19146475c9c09235399297fb4 P ab68d29fc55448dd826c71661e30a472 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0af6088cbad49b095e66ad2031d3b2a" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 37001 } } peers { permanent_uuid: "8388295b42914507b22f8a4099e79a02" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 42381 } } peers { permanent_uuid: "ab68d29fc55448dd826c71661e30a472" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 45573 } }
I20260811 09:08:15.672889 6544 consensus_queue.cc:260] T 49e696e19146475c9c09235399297fb4 P a0af6088cbad49b095e66ad2031d3b2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0af6088cbad49b095e66ad2031d3b2a" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 37001 } } peers { permanent_uuid: "8388295b42914507b22f8a4099e79a02" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 42381 } } peers { permanent_uuid: "ab68d29fc55448dd826c71661e30a472" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 45573 } }
I20260811 09:08:15.677137 6505 heartbeater.cc:503] Master 127.28.154.190:35113 was elected leader, sending a full tablet report...
I20260811 09:08:15.677821 6540 ts_tablet_manager.cc:1428] T 49e696e19146475c9c09235399297fb4 P ab68d29fc55448dd826c71661e30a472: Time spent starting tablet: real 0.018s user 0.010s sys 0.007s
I20260811 09:08:15.677812 6544 ts_tablet_manager.cc:1428] T 49e696e19146475c9c09235399297fb4 P a0af6088cbad49b095e66ad2031d3b2a: Time spent starting tablet: real 0.018s user 0.016s sys 0.000s
I20260811 09:08:15.754392 6554 raft_consensus.cc:482] T 49e696e19146475c9c09235399297fb4 P ab68d29fc55448dd826c71661e30a472 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:08:15.754705 6554 raft_consensus.cc:504] T 49e696e19146475c9c09235399297fb4 P ab68d29fc55448dd826c71661e30a472 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0af6088cbad49b095e66ad2031d3b2a" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 37001 } } peers { permanent_uuid: "8388295b42914507b22f8a4099e79a02" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 42381 } } peers { permanent_uuid: "ab68d29fc55448dd826c71661e30a472" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 45573 } }
I20260811 09:08:15.756412 6554 leader_election.cc:288] T 49e696e19146475c9c09235399297fb4 P ab68d29fc55448dd826c71661e30a472 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0af6088cbad49b095e66ad2031d3b2a (127.28.154.129:37001), 8388295b42914507b22f8a4099e79a02 (127.28.154.130:42381)
I20260811 09:08:15.759100 6534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:15.757036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.131:46673 (local address 127.28.154.130:42381)
0811 09:08:15.757387 (+ 351us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:15.757391 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:15.757419 (+ 28us) server_negotiation.cc:407] Connection header received
0811 09:08:15.757470 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:15.757473 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:15.757521 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:15.757607 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:15.758481 (+ 874us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:15.758493 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:15.758498 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:08:15.758521 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:15.758563 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:08:15.758581 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:15.758709 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:15.758799 (+ 90us) server_negotiation.cc:299] Negotiation successful
0811 09:08:15.758852 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":237}
I20260811 09:08:15.759111 6509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:15.756722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.130:42381 (local address 127.28.154.131:46673)
0811 09:08:15.757010 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:15.757030 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:15.757157 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:15.757666 (+ 509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:15.757671 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:15.757697 (+ 26us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:08:15.758253 (+ 556us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 09:08:15.758256 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:08:15.758273 (+ 17us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 09:08:15.758315 (+ 42us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:08:15.758691 (+ 376us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:08:15.758695 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:08:15.758698 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 09:08:15.758799 (+ 101us) client_negotiation.cc:240] Negotiation successful
0811 09:08:15.758853 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166}
I20260811 09:08:15.759461 6529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:15.756938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.131:42663 (local address 127.28.154.129:37001)
0811 09:08:15.757245 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:15.757251 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:15.757662 (+ 411us) server_negotiation.cc:407] Connection header received
0811 09:08:15.757815 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:15.757819 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:15.757868 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:15.757974 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:15.758804 (+ 830us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:15.758811 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:15.758815 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:08:15.758834 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:15.758872 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:08:15.758888 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:15.759001 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:15.759094 (+ 93us) server_negotiation.cc:299] Negotiation successful
0811 09:08:15.759154 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178}
I20260811 09:08:15.759662 6557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:15.756753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.129:37001 (local address 127.28.154.131:42663)
0811 09:08:15.757553 (+ 800us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:15.757568 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:15.757688 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:15.758000 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:15.758004 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:15.758027 (+ 23us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:08:15.758596 (+ 569us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 09:08:15.758599 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:08:15.758614 (+ 15us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 09:08:15.758657 (+ 43us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:08:15.758986 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:08:15.758991 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:08:15.758995 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 09:08:15.759116 (+ 121us) client_negotiation.cc:240] Negotiation successful
0811 09:08:15.759273 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":691,"thread_start_us":417,"threads_started":1}
I20260811 09:08:15.760309 6158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49e696e19146475c9c09235399297fb4" candidate_uuid: "ab68d29fc55448dd826c71661e30a472" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8388295b42914507b22f8a4099e79a02" is_pre_election: true
I20260811 09:08:15.760865 6158 raft_consensus.cc:2454] T 49e696e19146475c9c09235399297fb4 P 8388295b42914507b22f8a4099e79a02 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ab68d29fc55448dd826c71661e30a472 in term 0.
I20260811 09:08:15.760756 5912 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49e696e19146475c9c09235399297fb4" candidate_uuid: "ab68d29fc55448dd826c71661e30a472" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0af6088cbad49b095e66ad2031d3b2a" is_pre_election: true
I20260811 09:08:15.761242 5912 raft_consensus.cc:2454] T 49e696e19146475c9c09235399297fb4 P a0af6088cbad49b095e66ad2031d3b2a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ab68d29fc55448dd826c71661e30a472 in term 0.
I20260811 09:08:15.761555 6289 leader_election.cc:302] T 49e696e19146475c9c09235399297fb4 P ab68d29fc55448dd826c71661e30a472 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8388295b42914507b22f8a4099e79a02, ab68d29fc55448dd826c71661e30a472; no voters:
I20260811 09:08:15.762266 6554 raft_consensus.cc:2781] T 49e696e19146475c9c09235399297fb4 P ab68d29fc55448dd826c71661e30a472 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 09:08:15.762427 6554 raft_consensus.cc:482] T 49e696e19146475c9c09235399297fb4 P ab68d29fc55448dd826c71661e30a472 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 09:08:15.762521 6554 raft_consensus.cc:3037] T 49e696e19146475c9c09235399297fb4 P ab68d29fc55448dd826c71661e30a472 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:08:15.764767 6554 raft_consensus.cc:504] T 49e696e19146475c9c09235399297fb4 P ab68d29fc55448dd826c71661e30a472 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0af6088cbad49b095e66ad2031d3b2a" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 37001 } } peers { permanent_uuid: "8388295b42914507b22f8a4099e79a02" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 42381 } } peers { permanent_uuid: "ab68d29fc55448dd826c71661e30a472" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 45573 } }
I20260811 09:08:15.765354 6554 leader_election.cc:288] T 49e696e19146475c9c09235399297fb4 P ab68d29fc55448dd826c71661e30a472 [CANDIDATE]: Term 1 election: Requested vote from peers a0af6088cbad49b095e66ad2031d3b2a (127.28.154.129:37001), 8388295b42914507b22f8a4099e79a02 (127.28.154.130:42381)
I20260811 09:08:15.765883 5912 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49e696e19146475c9c09235399297fb4" candidate_uuid: "ab68d29fc55448dd826c71661e30a472" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0af6088cbad49b095e66ad2031d3b2a"
I20260811 09:08:15.765928 6158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49e696e19146475c9c09235399297fb4" candidate_uuid: "ab68d29fc55448dd826c71661e30a472" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8388295b42914507b22f8a4099e79a02"
I20260811 09:08:15.766124 5912 raft_consensus.cc:3037] T 49e696e19146475c9c09235399297fb4 P a0af6088cbad49b095e66ad2031d3b2a [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:08:15.766149 6158 raft_consensus.cc:3037] T 49e696e19146475c9c09235399297fb4 P 8388295b42914507b22f8a4099e79a02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:08:15.769462 6158 raft_consensus.cc:2454] T 49e696e19146475c9c09235399297fb4 P 8388295b42914507b22f8a4099e79a02 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ab68d29fc55448dd826c71661e30a472 in term 1.
I20260811 09:08:15.769462 5912 raft_consensus.cc:2454] T 49e696e19146475c9c09235399297fb4 P a0af6088cbad49b095e66ad2031d3b2a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ab68d29fc55448dd826c71661e30a472 in term 1.
I20260811 09:08:15.769956 6286 leader_election.cc:302] T 49e696e19146475c9c09235399297fb4 P ab68d29fc55448dd826c71661e30a472 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a0af6088cbad49b095e66ad2031d3b2a, ab68d29fc55448dd826c71661e30a472; no voters:
I20260811 09:08:15.770285 6554 raft_consensus.cc:2781] T 49e696e19146475c9c09235399297fb4 P ab68d29fc55448dd826c71661e30a472 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:08:15.770795 6554 raft_consensus.cc:686] T 49e696e19146475c9c09235399297fb4 P ab68d29fc55448dd826c71661e30a472 [term 1 LEADER]: Becoming Leader. State: Replica: ab68d29fc55448dd826c71661e30a472, State: Running, Role: LEADER
I20260811 09:08:15.771552 6554 consensus_queue.cc:237] T 49e696e19146475c9c09235399297fb4 P ab68d29fc55448dd826c71661e30a472 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0af6088cbad49b095e66ad2031d3b2a" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 37001 } } peers { permanent_uuid: "8388295b42914507b22f8a4099e79a02" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 42381 } } peers { permanent_uuid: "ab68d29fc55448dd826c71661e30a472" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 45573 } }
I20260811 09:08:15.776327 5659 catalog_manager.cc:5495] T 49e696e19146475c9c09235399297fb4 P ab68d29fc55448dd826c71661e30a472 reported cstate change: term changed from 0 to 1, leader changed from <none> to ab68d29fc55448dd826c71661e30a472 (127.28.154.131). New cstate: current_term: 1 leader_uuid: "ab68d29fc55448dd826c71661e30a472" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0af6088cbad49b095e66ad2031d3b2a" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 37001 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8388295b42914507b22f8a4099e79a02" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 42381 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ab68d29fc55448dd826c71661e30a472" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 45573 } health_report { overall_health: HEALTHY } } }
W20260811 09:08:15.807809 6508 tablet.cc:2369] T 49e696e19146475c9c09235399297fb4 P ab68d29fc55448dd826c71661e30a472: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 09:08:15.823351 6538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:15.821536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54472 (local address 127.28.154.131:45573)
0811 09:08:15.821767 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:15.821772 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:15.821793 (+ 21us) server_negotiation.cc:407] Connection header received
0811 09:08:15.821893 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:15.821897 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:15.821984 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:15.822078 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:15.822914 (+ 836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:15.822918 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:15.822921 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:08:15.822937 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:15.822970 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:08:15.822981 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:15.823080 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:15.823118 (+ 38us) server_negotiation.cc:299] Negotiation successful
0811 09:08:15.823162 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 09:08:15.831830 6158 raft_consensus.cc:1261] T 49e696e19146475c9c09235399297fb4 P 8388295b42914507b22f8a4099e79a02 [term 1 FOLLOWER]: Refusing update from remote peer ab68d29fc55448dd826c71661e30a472: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:08:15.831828 5912 raft_consensus.cc:1261] T 49e696e19146475c9c09235399297fb4 P a0af6088cbad49b095e66ad2031d3b2a [term 1 FOLLOWER]: Refusing update from remote peer ab68d29fc55448dd826c71661e30a472: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:08:15.832788 6558 consensus_queue.cc:1035] T 49e696e19146475c9c09235399297fb4 P ab68d29fc55448dd826c71661e30a472 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8388295b42914507b22f8a4099e79a02" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 42381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:08:15.833155 6554 consensus_queue.cc:1035] T 49e696e19146475c9c09235399297fb4 P ab68d29fc55448dd826c71661e30a472 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0af6088cbad49b095e66ad2031d3b2a" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 37001 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:08:15.848618 6566 mvcc.cc:205] Tried to move back new op lower bound from 7317255355717095424 to 7317255355484798976. Current Snapshot: MvccSnapshot[applied={T|T < 7317255355717095424}]
I20260811 09:08:15.860670 5659 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44026:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:08:15.860967 5659 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44026:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:08:15.865140 5659 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 274f21b7412140d0a093422fdd96c125: Sending DeleteTablet for 3 replicas of tablet 49e696e19146475c9c09235399297fb4
I20260811 09:08:15.866416 6379 tablet_service.cc:1514] Processing DeleteTablet for tablet 49e696e19146475c9c09235399297fb4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:08:15 UTC) from {username='slave'} at 127.0.0.1:54466
I20260811 09:08:15.866466 6118 tablet_service.cc:1514] Processing DeleteTablet for tablet 49e696e19146475c9c09235399297fb4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:08:15 UTC) from {username='slave'} at 127.0.0.1:55114
I20260811 09:08:15.866521 5883 tablet_service.cc:1514] Processing DeleteTablet for tablet 49e696e19146475c9c09235399297fb4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:08:15 UTC) from {username='slave'} at 127.0.0.1:35710
I20260811 09:08:15.870275 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 5766
I20260811 09:08:15.877590 6582 tablet_replica.cc:330] T 49e696e19146475c9c09235399297fb4 P a0af6088cbad49b095e66ad2031d3b2a: stopping tablet replica
I20260811 09:08:15.878175 6582 raft_consensus.cc:2229] T 49e696e19146475c9c09235399297fb4 P a0af6088cbad49b095e66ad2031d3b2a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:08:15.878779 6582 raft_consensus.cc:2258] T 49e696e19146475c9c09235399297fb4 P a0af6088cbad49b095e66ad2031d3b2a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:08:15.881311 6581 tablet_replica.cc:330] T 49e696e19146475c9c09235399297fb4 P ab68d29fc55448dd826c71661e30a472: stopping tablet replica
I20260811 09:08:15.881847 6581 raft_consensus.cc:2229] T 49e696e19146475c9c09235399297fb4 P ab68d29fc55448dd826c71661e30a472 [term 1 LEADER]: Raft consensus shutting down.
I20260811 09:08:15.882661 6581 raft_consensus.cc:2258] T 49e696e19146475c9c09235399297fb4 P ab68d29fc55448dd826c71661e30a472 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:08:15.883462 6578 tablet_replica.cc:330] T 49e696e19146475c9c09235399297fb4 P 8388295b42914507b22f8a4099e79a02: stopping tablet replica
I20260811 09:08:15.883805 6578 raft_consensus.cc:2229] T 49e696e19146475c9c09235399297fb4 P 8388295b42914507b22f8a4099e79a02 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:08:15.884240 6578 raft_consensus.cc:2258] T 49e696e19146475c9c09235399297fb4 P 8388295b42914507b22f8a4099e79a02 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:08:15.886643 6582 ts_tablet_manager.cc:1905] T 49e696e19146475c9c09235399297fb4 P a0af6088cbad49b095e66ad2031d3b2a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:08:15.887224 6578 ts_tablet_manager.cc:1905] T 49e696e19146475c9c09235399297fb4 P 8388295b42914507b22f8a4099e79a02: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:08:15.889640 6529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:15.871989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35716 (local address 127.28.154.129:37001)
0811 09:08:15.872214 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:15.872221 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:15.884397 (+ 12176us) server_negotiation.cc:407] Connection header received
0811 09:08:15.884968 (+ 571us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:15.884974 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:15.885181 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:15.885290 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:15.888851 (+ 3561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:15.888857 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:15.888862 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:08:15.888885 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:15.888926 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:08:15.888942 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:15.889040 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:15.889266 (+ 226us) server_negotiation.cc:299] Negotiation successful
0811 09:08:15.889332 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 09:08:15.891105 5987 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:08:15.895099 6578 ts_tablet_manager.cc:1918] T 49e696e19146475c9c09235399297fb4 P 8388295b42914507b22f8a4099e79a02: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:08:15.895267 6578 log.cc:1196] T 49e696e19146475c9c09235399297fb4 P 8388295b42914507b22f8a4099e79a02: Deleting WAL directory at /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439250192320-29290-0/minicluster-data/ts-1/wal/wals/49e696e19146475c9c09235399297fb4
I20260811 09:08:15.895700 6578 ts_tablet_manager.cc:1939] T 49e696e19146475c9c09235399297fb4 P 8388295b42914507b22f8a4099e79a02: Deleting consensus metadata
I20260811 09:08:15.897107 6581 ts_tablet_manager.cc:1905] T 49e696e19146475c9c09235399297fb4 P ab68d29fc55448dd826c71661e30a472: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:08:15.897620 5632 catalog_manager.cc:4841] TS 8388295b42914507b22f8a4099e79a02 (127.28.154.130:42381): tablet 49e696e19146475c9c09235399297fb4 (table test-table [id=11d25002158d475994bb65ec6035dd39]) successfully deleted
I20260811 09:08:15.902961 6581 ts_tablet_manager.cc:1918] T 49e696e19146475c9c09235399297fb4 P ab68d29fc55448dd826c71661e30a472: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:08:15.903155 6581 log.cc:1196] T 49e696e19146475c9c09235399297fb4 P ab68d29fc55448dd826c71661e30a472: Deleting WAL directory at /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786439250192320-29290-0/minicluster-data/ts-2/wal/wals/49e696e19146475c9c09235399297fb4
I20260811 09:08:15.903643 6581 ts_tablet_manager.cc:1939] T 49e696e19146475c9c09235399297fb4 P ab68d29fc55448dd826c71661e30a472: Deleting consensus metadata
I20260811 09:08:15.905329 5634 catalog_manager.cc:4841] TS ab68d29fc55448dd826c71661e30a472 (127.28.154.131:45573): tablet 49e696e19146475c9c09235399297fb4 (table test-table [id=11d25002158d475994bb65ec6035dd39]) successfully deleted
I20260811 09:08:16.003176 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 5766
W20260811 09:08:16.031915 5627 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.28.154.129:37001 (error 108)
I20260811 09:08:16.032104 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 6002
I20260811 09:08:16.034201 6531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:16.033611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.129:37001 (local address 127.0.0.1:35732)
0811 09:08:16.033820 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:16.034033 (+ 213us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.129:37001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260811 09:08:16.034469 5627 catalog_manager.cc:4584] TS a0af6088cbad49b095e66ad2031d3b2a (127.28.154.129:37001): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 49e696e19146475c9c09235399297fb4: Network error: Client connection negotiation failed: client connection to 127.28.154.129:37001: connect: Connection refused (error 111)
I20260811 09:08:16.035444 6534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:16.033335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55124 (local address 127.28.154.130:42381)
0811 09:08:16.033544 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:16.033550 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:16.033569 (+ 19us) server_negotiation.cc:407] Connection header received
0811 09:08:16.033632 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:16.033637 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:16.033752 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:16.033854 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:16.034760 (+ 906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:16.034766 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:16.034770 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:08:16.034789 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:16.034831 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:08:16.034847 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:16.034965 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:16.035060 (+ 95us) server_negotiation.cc:299] Negotiation successful
0811 09:08:16.035125 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 09:08:16.036568 6248 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:08:16.088584 6531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:16.087945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.129:37001 (local address 127.0.0.1:35734)
0811 09:08:16.088233 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:16.088365 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.129:37001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162}
I20260811 09:08:16.089881 6531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:16.089432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.129:37001 (local address 127.0.0.1:35736)
0811 09:08:16.089597 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:16.089707 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.129:37001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260811 09:08:16.143453 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 6002
I20260811 09:08:16.160491 6531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:16.159836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.129:37001 (local address 127.0.0.1:35752)
0811 09:08:16.160107 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:16.160229 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.129:37001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260811 09:08:16.161736 6531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:16.161313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.129:37001 (local address 127.0.0.1:35766)
0811 09:08:16.161483 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:16.161582 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.129:37001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260811 09:08:16.175776 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 6263
I20260811 09:08:16.179196 6538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:16.176875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54478 (local address 127.28.154.131:45573)
0811 09:08:16.177039 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:16.177043 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:16.177063 (+ 20us) server_negotiation.cc:407] Connection header received
0811 09:08:16.177260 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:16.177266 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:16.177327 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:16.177418 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:16.178498 (+ 1080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:16.178502 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:16.178506 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:08:16.178528 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:16.178566 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:08:16.178579 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:16.178701 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:16.178818 (+ 117us) server_negotiation.cc:299] Negotiation successful
0811 09:08:16.178875 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 09:08:16.180028 6495 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:08:16.242745 6531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:16.242185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.129:37001 (local address 127.0.0.1:35770)
0811 09:08:16.242438 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:16.242547 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.129:37001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
I20260811 09:08:16.243818 6531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:16.243426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.129:37001 (local address 127.0.0.1:35774)
0811 09:08:16.243628 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:16.243699 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.129:37001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260811 09:08:16.303678 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 6263
I20260811 09:08:16.330538 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 5609
I20260811 09:08:16.331782 5713 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:08:16.439811 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 5609
I20260811 09:08:16.439945 6531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:16.439467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.129:37001 (local address 127.0.0.1:35790)
0811 09:08:16.439653 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:16.439758 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.129:37001: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
2026-08-11T09:08:16Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (2907 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260811 09:08:16.487509 29290 mini_oidc.cc:152] Starting JWKS server
I20260811 09:08:16.491786 29290 webserver.cc:460] Webserver started at https://127.0.0.1:35301/ using document root <none> and password file <none>
I20260811 09:08:16.492141 29290 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 09:08:16.492853 29290 webserver.cc:460] Webserver started at http://127.0.0.1:45335/ using document root <none> and password file <none>
2026-08-11T09:08:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:08:16Z Disabled control of system clock
I20260811 09:08:16.525543 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786439250192320-29290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786439250192320-29290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786439250192320-29290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.154.190:34193
--webserver_interface=127.28.154.190
--webserver_port=0
--builtin_ntp_servers=127.28.154.148:41395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.154.190:34193
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35301/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786439250192320-29290-0/testchainca.cert with env {}
W20260811 09:08:16.862452 6599 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:08:16.862766 6599 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:08:16.862852 6599 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:08:16.870158 6599 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:08:16.870285 6599 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:08:16.870322 6599 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:08:16.870383 6599 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:08:16.870426 6599 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 09:08:16.870498 6599 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 09:08:16.870537 6599 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:35301/jwks.json
I20260811 09:08:16.881286 6599 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:41395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786439250192320-29290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786439250192320-29290-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.154.190:34193
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.154.190:34193
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35301/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.154.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786439250192320-29290-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786439250192320-29290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:08:16.882628 6599 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:08:16.883869 6599 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:08:16.895373 6606 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:08:16.895313 6610 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:08:16.895396 6607 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:08:16.896281 6599 server_base.cc:1034] running on GCE node
I20260811 09:08:16.897063 6599 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:08:16.898447 6599 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:08:16.899677 6599 hybrid_clock.cc:648] HybridClock initialized: now 1786439296899605 us; error 55 us; skew 500 ppm
I20260811 09:08:16.900007 6599 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 09:08:16.902308 6599 webserver.cc:460] Webserver started at http://127.28.154.190:42119/ using document root <none> and password file <none>
I20260811 09:08:16.902959 6599 fs_manager.cc:359] Metadata directory not provided
I20260811 09:08:16.903059 6599 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:08:16.903323 6599 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:08:16.906191 6599 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786439250192320-29290-0/minicluster-data/master-0/data/instance:
uuid: "c20a135058134454b632a1a012a1c44a"
format_stamp: "Formatted at 2026-08-11 09:08:16 on dist-test-slave-xt4k"
server_key: "09a896621f9ad0d9f74af2d4d27df92f"
server_key_iv: "e99295d9e2f3053659eec0e7247c6441"
server_key_version: "encryptionkey@0"
I20260811 09:08:16.906919 6599 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786439250192320-29290-0/minicluster-data/master-0/wal/instance:
uuid: "c20a135058134454b632a1a012a1c44a"
format_stamp: "Formatted at 2026-08-11 09:08:16 on dist-test-slave-xt4k"
server_key: "09a896621f9ad0d9f74af2d4d27df92f"
server_key_iv: "e99295d9e2f3053659eec0e7247c6441"
server_key_version: "encryptionkey@0"
I20260811 09:08:16.911921 6599 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 09:08:16.915386 6625 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:08:16.916989 6599 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 09:08:16.917198 6599 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786439250192320-29290-0/minicluster-data/master-0/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786439250192320-29290-0/minicluster-data/master-0/wal
uuid: "c20a135058134454b632a1a012a1c44a"
format_stamp: "Formatted at 2026-08-11 09:08:16 on dist-test-slave-xt4k"
server_key: "09a896621f9ad0d9f74af2d4d27df92f"
server_key_iv: "e99295d9e2f3053659eec0e7247c6441"
server_key_version: "encryptionkey@0"
I20260811 09:08:16.917423 6599 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786439250192320-29290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786439250192320-29290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786439250192320-29290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:08:16.936372 6599 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:08:16.937247 6599 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:08:16.937558 6599 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:08:16.955197 6599 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.190:34193
I20260811 09:08:16.955217 6719 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.190:34193 every 8 connection(s)
I20260811 09:08:16.956631 6599 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
I20260811 09:08:16.958257 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 6599
I20260811 09:08:16.958462 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786439250192320-29290-0/minicluster-data/master-0/wal/instance
I20260811 09:08:16.959259 29290 external_mini_cluster.cc:1428] Setting key 2382bc4835b0faf3dd60d8fef857d305
I20260811 09:08:16.961433 6720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:08:16.973412 6725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:16.961252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37250 (local address 127.28.154.190:34193)
0811 09:08:16.962243 (+ 991us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:16.962254 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:16.962297 (+ 43us) server_negotiation.cc:407] Connection header received
0811 09:08:16.963923 (+ 1626us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:16.963946 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:16.964476 (+ 530us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:16.964852 (+ 376us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:16.965534 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:16.966933 (+ 1399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:16.968859 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:16.969515 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:16.969699 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:16.970969 (+ 1270us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:16.970986 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:16.971008 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:08:16.971584 (+ 576us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:16.971675 (+ 91us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:08:16.971698 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:16.971795 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:16.972026 (+ 231us) server_negotiation.cc:299] Negotiation successful
0811 09:08:16.972404 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":584,"thread_start_us":378,"threads_started":1}
I20260811 09:08:16.974359 6720 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c20a135058134454b632a1a012a1c44a: Bootstrap starting.
I20260811 09:08:16.977175 29290 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 09:08:16.977643 6720 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c20a135058134454b632a1a012a1c44a: Neither blocks nor log segments found. Creating new log.
I20260811 09:08:16.978834 6720 log.cc:824] T 00000000000000000000000000000000 P c20a135058134454b632a1a012a1c44a: Log is configured to *not* fsync() on all Append() calls
I20260811 09:08:16.981829 6720 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c20a135058134454b632a1a012a1c44a: No bootstrap required, opened a new log
I20260811 09:08:16.989128 6720 raft_consensus.cc:348] T 00000000000000000000000000000000 P c20a135058134454b632a1a012a1c44a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c20a135058134454b632a1a012a1c44a" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 34193 } }
I20260811 09:08:16.989506 6720 raft_consensus.cc:374] T 00000000000000000000000000000000 P c20a135058134454b632a1a012a1c44a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:08:16.989584 6720 raft_consensus.cc:729] T 00000000000000000000000000000000 P c20a135058134454b632a1a012a1c44a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c20a135058134454b632a1a012a1c44a, State: Initialized, Role: FOLLOWER
I20260811 09:08:16.990173 6720 consensus_queue.cc:260] T 00000000000000000000000000000000 P c20a135058134454b632a1a012a1c44a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c20a135058134454b632a1a012a1c44a" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 34193 } }
I20260811 09:08:16.990464 6720 raft_consensus.cc:388] T 00000000000000000000000000000000 P c20a135058134454b632a1a012a1c44a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:08:16.990650 6720 raft_consensus.cc:482] T 00000000000000000000000000000000 P c20a135058134454b632a1a012a1c44a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:08:16.990833 6720 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c20a135058134454b632a1a012a1c44a [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:08:16.993067 6720 raft_consensus.cc:504] T 00000000000000000000000000000000 P c20a135058134454b632a1a012a1c44a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c20a135058134454b632a1a012a1c44a" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 34193 } }
I20260811 09:08:16.993609 6720 leader_election.cc:302] T 00000000000000000000000000000000 P c20a135058134454b632a1a012a1c44a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c20a135058134454b632a1a012a1c44a; no voters:
I20260811 09:08:16.994200 6720 leader_election.cc:288] T 00000000000000000000000000000000 P c20a135058134454b632a1a012a1c44a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:08:16.994338 6729 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c20a135058134454b632a1a012a1c44a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:08:16.995048 6729 raft_consensus.cc:686] T 00000000000000000000000000000000 P c20a135058134454b632a1a012a1c44a [term 1 LEADER]: Becoming Leader. State: Replica: c20a135058134454b632a1a012a1c44a, State: Running, Role: LEADER
I20260811 09:08:16.995648 6729 consensus_queue.cc:237] T 00000000000000000000000000000000 P c20a135058134454b632a1a012a1c44a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c20a135058134454b632a1a012a1c44a" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 34193 } }
I20260811 09:08:16.996309 6720 sys_catalog.cc:564] T 00000000000000000000000000000000 P c20a135058134454b632a1a012a1c44a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:08:16.998579 6731 sys_catalog.cc:455] T 00000000000000000000000000000000 P c20a135058134454b632a1a012a1c44a [sys.catalog]: SysCatalogTable state changed. Reason: New leader c20a135058134454b632a1a012a1c44a. Latest consensus state: current_term: 1 leader_uuid: "c20a135058134454b632a1a012a1c44a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c20a135058134454b632a1a012a1c44a" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 34193 } } }
I20260811 09:08:16.999385 6731 sys_catalog.cc:458] T 00000000000000000000000000000000 P c20a135058134454b632a1a012a1c44a [sys.catalog]: This master's current role is: LEADER
I20260811 09:08:16.999094 6730 sys_catalog.cc:455] T 00000000000000000000000000000000 P c20a135058134454b632a1a012a1c44a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c20a135058134454b632a1a012a1c44a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c20a135058134454b632a1a012a1c44a" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 34193 } } }
I20260811 09:08:17.000048 6730 sys_catalog.cc:458] T 00000000000000000000000000000000 P c20a135058134454b632a1a012a1c44a [sys.catalog]: This master's current role is: LEADER
I20260811 09:08:17.001034 6742 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:08:17.006440 6742 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:08:17.016206 6742 catalog_manager.cc:1347] Generated new cluster ID: f67c0f9fdfe34cd781934f63090f0f7b
I20260811 09:08:17.016376 6742 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:08:17.027765 6742 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:08:17.029845 6742 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:08:17.038381 6742 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c20a135058134454b632a1a012a1c44a: Generated new TSK 0
I20260811 09:08:17.039290 6742 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260811 09:08:17.086621 6784 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260811 09:08:17.086879 6784 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 09:08:17.089064 6725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:17.064696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37252 (local address 127.28.154.190:34193)
0811 09:08:17.064929 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:17.064937 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:17.065394 (+ 457us) server_negotiation.cc:407] Connection header received
0811 09:08:17.065830 (+ 436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:17.065835 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:17.065914 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:17.066037 (+ 123us) server_negotiation.cc:226] Negotiated authn=JWT
0811 09:08:17.066702 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:17.067457 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:17.069630 (+ 2173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:17.070099 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:17.070196 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:17.070872 (+ 676us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 09:08:17.088266 (+ 17394us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35301/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0811 09:08:17.088797 (+ 531us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:37252: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35301/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":76}
W20260811 09:08:17.089195 6725 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:37252: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35301/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260811 09:08:17.091320 6782 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:08:17.064583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:34193 (local address 127.0.0.1:37252)
0811 09:08:17.065277 (+ 694us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:17.065299 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:17.065501 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:17.066099 (+ 598us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:17.066105 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:17.066119 (+ 14us) client_negotiation.cc:189] Negotiated authn=JWT
0811 09:08:17.066526 (+ 407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:17.066538 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:17.067631 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:17.067638 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:17.069472 (+ 1834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:17.069488 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:17.070284 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:17.070291 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:17.070483 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:17.070575 (+ 92us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 09:08:17.088662 (+ 18087us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35301/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0811 09:08:17.089175 (+ 513us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.28.154.190:34193: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35301/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":329,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.28.154.190:34193: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35301/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260811 09:08:17.092729 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 6599
I20260811 09:08:17.093888 6712 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:08:17.190595 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 6599
2026-08-11T09:08:17Z chronyd exiting
I20260811 09:08:17.246527 29290 test_util.cc:175] -----------------------------------------------
I20260811 09:08:17.246727 29290 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786439250192320-29290-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (764 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260811 09:08:17.250789 29290 mini_oidc.cc:152] Starting JWKS server
I20260811 09:08:17.252640 29290 webserver.cc:460] Webserver started at https://127.0.0.1:36517/ using document root <none> and password file <none>
I20260811 09:08:17.252863 29290 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 09:08:17.253475 29290 webserver.cc:460] Webserver started at http://127.0.0.1:36409/ using document root <none> and password file <none>
2026-08-11T09:08:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:08:17Z Disabled control of system clock
I20260811 09:08:17.287153 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786439250192320-29290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786439250192320-29290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786439250192320-29290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.154.190:39703
--webserver_interface=127.28.154.190
--webserver_port=0
--builtin_ntp_servers=127.28.154.148:43555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.154.190:39703
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36517/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786439250192320-29290-0/testchainca.cert with env {}
W20260811 09:08:17.626474 6794 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:08:17.626915 6794 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:08:17.627075 6794 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:08:17.634555 6794 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:08:17.634675 6794 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:08:17.634713 6794 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:08:17.634769 6794 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:08:17.634814 6794 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 09:08:17.634873 6794 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 09:08:17.634912 6794 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:36517/jwks.json
I20260811 09:08:17.645342 6794 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:43555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786439250192320-29290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786439250192320-29290-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.154.190:39703
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.154.190:39703
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36517/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.154.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786439250192320-29290-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786439250192320-29290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:08:17.646848 6794 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:08:17.648169 6794 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:08:17.660403 6806 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:08:17.660485 6813 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:08:17.660418 6808 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:08:17.661083 6794 server_base.cc:1034] running on GCE node
I20260811 09:08:17.661906 6794 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:08:17.663343 6794 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:08:17.664553 6794 hybrid_clock.cc:648] HybridClock initialized: now 1786439297664504 us; error 42 us; skew 500 ppm
I20260811 09:08:17.664908 6794 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 09:08:17.667250 6794 webserver.cc:460] Webserver started at http://127.28.154.190:33631/ using document root <none> and password file <none>
I20260811 09:08:17.667975 6794 fs_manager.cc:359] Metadata directory not provided
I20260811 09:08:17.668085 6794 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:08:17.668440 6794 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:08:17.671882 6794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786439250192320-29290-0/minicluster-data/master-0/data/instance:
uuid: "fd922be5dd714b2ab39faadc7cce4fd1"
format_stamp: "Formatted at 2026-08-11 09:08:17 on dist-test-slave-xt4k"
server_key: "ebef2d82a1ef9ab39d125c830a9e3a1f"
server_key_iv: "355bd54ca6326453430ed8e6cdb8f41f"
server_key_version: "encryptionkey@0"
I20260811 09:08:17.672842 6794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786439250192320-29290-0/minicluster-data/master-0/wal/instance:
uuid: "fd922be5dd714b2ab39faadc7cce4fd1"
format_stamp: "Formatted at 2026-08-11 09:08:17 on dist-test-slave-xt4k"
server_key: "ebef2d82a1ef9ab39d125c830a9e3a1f"
server_key_iv: "355bd54ca6326453430ed8e6cdb8f41f"
server_key_version: "encryptionkey@0"
I20260811 09:08:17.678239 6794 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.002s
I20260811 09:08:17.682132 6824 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:08:17.683789 6794 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 09:08:17.683995 6794 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786439250192320-29290-0/minicluster-data/master-0/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786439250192320-29290-0/minicluster-data/master-0/wal
uuid: "fd922be5dd714b2ab39faadc7cce4fd1"
format_stamp: "Formatted at 2026-08-11 09:08:17 on dist-test-slave-xt4k"
server_key: "ebef2d82a1ef9ab39d125c830a9e3a1f"
server_key_iv: "355bd54ca6326453430ed8e6cdb8f41f"
server_key_version: "encryptionkey@0"
I20260811 09:08:17.684242 6794 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786439250192320-29290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786439250192320-29290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786439250192320-29290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:08:17.703322 6794 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:08:17.704268 6794 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:08:17.704613 6794 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:08:17.723698 6794 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.190:39703
I20260811 09:08:17.723717 6924 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.190:39703 every 8 connection(s)
I20260811 09:08:17.725211 6794 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
I20260811 09:08:17.729462 6925 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:08:17.732393 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 6794
I20260811 09:08:17.732633 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786439250192320-29290-0/minicluster-data/master-0/wal/instance
I20260811 09:08:17.733448 29290 external_mini_cluster.cc:1428] Setting key c1c507a88bc5b099b73876a920b41035
I20260811 09:08:17.742594 6925 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd922be5dd714b2ab39faadc7cce4fd1: Bootstrap starting.
I20260811 09:08:17.745909 6925 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fd922be5dd714b2ab39faadc7cce4fd1: Neither blocks nor log segments found. Creating new log.
I20260811 09:08:17.747089 6925 log.cc:824] T 00000000000000000000000000000000 P fd922be5dd714b2ab39faadc7cce4fd1: Log is configured to *not* fsync() on all Append() calls
I20260811 09:08:17.747953 6935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:17.735941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36410 (local address 127.28.154.190:39703)
0811 09:08:17.737122 (+ 1181us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:17.737135 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:17.737182 (+ 47us) server_negotiation.cc:407] Connection header received
0811 09:08:17.738954 (+ 1772us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:17.738982 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:17.739547 (+ 565us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:17.739961 (+ 414us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:17.740629 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:17.741919 (+ 1290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:17.743278 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:17.744002 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:17.744199 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:17.745384 (+ 1185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:17.745392 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:17.745407 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:08:17.745953 (+ 546us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:17.746051 (+ 98us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:08:17.746091 (+ 40us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:17.746216 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:17.746444 (+ 228us) server_negotiation.cc:299] Negotiation successful
0811 09:08:17.746763 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":690,"thread_start_us":455,"threads_started":1}
I20260811 09:08:17.750558 6925 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd922be5dd714b2ab39faadc7cce4fd1: No bootstrap required, opened a new log
I20260811 09:08:17.752123 29290 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 09:08:17.758752 6925 raft_consensus.cc:348] T 00000000000000000000000000000000 P fd922be5dd714b2ab39faadc7cce4fd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd922be5dd714b2ab39faadc7cce4fd1" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 39703 } }
I20260811 09:08:17.759161 6925 raft_consensus.cc:374] T 00000000000000000000000000000000 P fd922be5dd714b2ab39faadc7cce4fd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:08:17.759266 6925 raft_consensus.cc:729] T 00000000000000000000000000000000 P fd922be5dd714b2ab39faadc7cce4fd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd922be5dd714b2ab39faadc7cce4fd1, State: Initialized, Role: FOLLOWER
I20260811 09:08:17.759862 6925 consensus_queue.cc:260] T 00000000000000000000000000000000 P fd922be5dd714b2ab39faadc7cce4fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd922be5dd714b2ab39faadc7cce4fd1" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 39703 } }
I20260811 09:08:17.760125 6925 raft_consensus.cc:388] T 00000000000000000000000000000000 P fd922be5dd714b2ab39faadc7cce4fd1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:08:17.760275 6925 raft_consensus.cc:482] T 00000000000000000000000000000000 P fd922be5dd714b2ab39faadc7cce4fd1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:08:17.760548 6925 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fd922be5dd714b2ab39faadc7cce4fd1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:08:17.762883 6925 raft_consensus.cc:504] T 00000000000000000000000000000000 P fd922be5dd714b2ab39faadc7cce4fd1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd922be5dd714b2ab39faadc7cce4fd1" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 39703 } }
I20260811 09:08:17.763410 6925 leader_election.cc:302] T 00000000000000000000000000000000 P fd922be5dd714b2ab39faadc7cce4fd1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fd922be5dd714b2ab39faadc7cce4fd1; no voters:
I20260811 09:08:17.764027 6925 leader_election.cc:288] T 00000000000000000000000000000000 P fd922be5dd714b2ab39faadc7cce4fd1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:08:17.764174 6937 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fd922be5dd714b2ab39faadc7cce4fd1 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:08:17.764865 6937 raft_consensus.cc:686] T 00000000000000000000000000000000 P fd922be5dd714b2ab39faadc7cce4fd1 [term 1 LEADER]: Becoming Leader. State: Replica: fd922be5dd714b2ab39faadc7cce4fd1, State: Running, Role: LEADER
I20260811 09:08:17.765385 6937 consensus_queue.cc:237] T 00000000000000000000000000000000 P fd922be5dd714b2ab39faadc7cce4fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd922be5dd714b2ab39faadc7cce4fd1" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 39703 } }
I20260811 09:08:17.766103 6925 sys_catalog.cc:564] T 00000000000000000000000000000000 P fd922be5dd714b2ab39faadc7cce4fd1 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:08:17.768016 6938 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd922be5dd714b2ab39faadc7cce4fd1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fd922be5dd714b2ab39faadc7cce4fd1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd922be5dd714b2ab39faadc7cce4fd1" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 39703 } } }
I20260811 09:08:17.768493 6938 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd922be5dd714b2ab39faadc7cce4fd1 [sys.catalog]: This master's current role is: LEADER
I20260811 09:08:17.768988 6939 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd922be5dd714b2ab39faadc7cce4fd1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fd922be5dd714b2ab39faadc7cce4fd1. Latest consensus state: current_term: 1 leader_uuid: "fd922be5dd714b2ab39faadc7cce4fd1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd922be5dd714b2ab39faadc7cce4fd1" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 39703 } } }
I20260811 09:08:17.769299 6939 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd922be5dd714b2ab39faadc7cce4fd1 [sys.catalog]: This master's current role is: LEADER
I20260811 09:08:17.771153 6951 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:08:17.776690 6951 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:08:17.787259 6951 catalog_manager.cc:1347] Generated new cluster ID: 8f89a5d039044f978ed80ed2c9ffc6a8
I20260811 09:08:17.787384 6951 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:08:17.800034 6951 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:08:17.801432 6951 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:08:17.814956 6951 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fd922be5dd714b2ab39faadc7cce4fd1: Generated new TSK 0
I20260811 09:08:17.816072 6951 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260811 09:08:17.855391 6989 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260811 09:08:17.855536 6989 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 09:08:17.856523 6935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:17.828478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36412 (local address 127.28.154.190:39703)
0811 09:08:17.828704 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:17.828711 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:17.829109 (+ 398us) server_negotiation.cc:407] Connection header received
0811 09:08:17.829355 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:17.829359 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:17.829435 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:17.829604 (+ 169us) server_negotiation.cc:226] Negotiated authn=JWT
0811 09:08:17.830210 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:17.830986 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:17.832422 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:17.832911 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:17.833039 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:17.833535 (+ 496us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 09:08:17.855888 (+ 22353us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36517/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0811 09:08:17.856239 (+ 351us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:36412: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36517/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":56}
W20260811 09:08:17.856652 6935 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:36412: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36517/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260811 09:08:17.856895 6985 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:08:17.828386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:39703 (local address 127.0.0.1:36412)
0811 09:08:17.828989 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:17.829009 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:17.829167 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:17.829633 (+ 466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:17.829639 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:17.829651 (+ 12us) client_negotiation.cc:189] Negotiated authn=JWT
0811 09:08:17.830022 (+ 371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:17.830034 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:17.831156 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:17.831161 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:17.832258 (+ 1097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:17.832270 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:17.833091 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:17.833097 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:17.833245 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:17.833282 (+ 37us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 09:08:17.856171 (+ 22889us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36517/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0811 09:08:17.856504 (+ 333us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.28.154.190:39703: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36517/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":358,"threads_started":1}
I20260811 09:08:17.858467 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 6794
I20260811 09:08:17.859831 6918 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:08:17.952747 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 6794
2026-08-11T09:08:17Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (767 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260811 09:08:18.017853 29290 mini_oidc.cc:152] Starting JWKS server
I20260811 09:08:18.019788 29290 webserver.cc:460] Webserver started at https://127.0.0.1:42279/ using document root <none> and password file <none>
I20260811 09:08:18.020015 29290 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 09:08:18.020725 29290 webserver.cc:460] Webserver started at http://127.0.0.1:43165/ using document root <none> and password file <none>
2026-08-11T09:08:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:08:18Z Disabled control of system clock
I20260811 09:08:18.054199 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786439250192320-29290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786439250192320-29290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786439250192320-29290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.154.190:46121
--webserver_interface=127.28.154.190
--webserver_port=0
--builtin_ntp_servers=127.28.154.148:44921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.154.190:46121
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42279/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260811 09:08:18.394255 7006 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:08:18.394565 7006 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:08:18.394682 7006 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:08:18.401953 7006 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:08:18.402069 7006 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:08:18.402107 7006 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:08:18.402176 7006 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:08:18.402238 7006 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 09:08:18.402320 7006 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 09:08:18.402380 7006 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:42279/jwks.json
I20260811 09:08:18.411768 7006 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:44921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786439250192320-29290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786439250192320-29290-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.154.190:46121
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.154.190:46121
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42279/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.154.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786439250192320-29290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:08:18.413117 7006 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:08:18.414935 7006 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:08:18.427495 7015 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:08:18.427434 7016 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:08:18.427762 7019 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:08:18.429006 7006 server_base.cc:1034] running on GCE node
I20260811 09:08:18.429872 7006 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:08:18.431383 7006 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:08:18.432587 7006 hybrid_clock.cc:648] HybridClock initialized: now 1786439298432532 us; error 50 us; skew 500 ppm
I20260811 09:08:18.432935 7006 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 09:08:18.435247 7006 webserver.cc:460] Webserver started at http://127.28.154.190:43029/ using document root <none> and password file <none>
I20260811 09:08:18.436121 7006 fs_manager.cc:359] Metadata directory not provided
I20260811 09:08:18.436237 7006 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:08:18.436606 7006 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:08:18.439581 7006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786439250192320-29290-0/minicluster-data/master-0/data/instance:
uuid: "d95023fbc2114766af2c638add98503a"
format_stamp: "Formatted at 2026-08-11 09:08:18 on dist-test-slave-xt4k"
server_key: "94c26c0aadc4a1520760890929168055"
server_key_iv: "ea00345d0e515321a72e9a8000f1884b"
server_key_version: "encryptionkey@0"
I20260811 09:08:18.440413 7006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786439250192320-29290-0/minicluster-data/master-0/wal/instance:
uuid: "d95023fbc2114766af2c638add98503a"
format_stamp: "Formatted at 2026-08-11 09:08:18 on dist-test-slave-xt4k"
server_key: "94c26c0aadc4a1520760890929168055"
server_key_iv: "ea00345d0e515321a72e9a8000f1884b"
server_key_version: "encryptionkey@0"
I20260811 09:08:18.446014 7006 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 09:08:18.449764 7029 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:08:18.451205 7006 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 09:08:18.451398 7006 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786439250192320-29290-0/minicluster-data/master-0/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786439250192320-29290-0/minicluster-data/master-0/wal
uuid: "d95023fbc2114766af2c638add98503a"
format_stamp: "Formatted at 2026-08-11 09:08:18 on dist-test-slave-xt4k"
server_key: "94c26c0aadc4a1520760890929168055"
server_key_iv: "ea00345d0e515321a72e9a8000f1884b"
server_key_version: "encryptionkey@0"
I20260811 09:08:18.451618 7006 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786439250192320-29290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786439250192320-29290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786439250192320-29290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:08:18.474074 7006 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:08:18.475240 7006 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:08:18.475574 7006 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:08:18.495697 7006 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.190:46121
I20260811 09:08:18.495715 7136 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.190:46121 every 8 connection(s)
I20260811 09:08:18.497164 7006 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
I20260811 09:08:18.501366 7137 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:08:18.507638 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 7006
I20260811 09:08:18.507835 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786439250192320-29290-0/minicluster-data/master-0/wal/instance
I20260811 09:08:18.508687 29290 external_mini_cluster.cc:1428] Setting key bee8462087ee8b782d4aa323033caa7f
I20260811 09:08:18.513441 7137 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d95023fbc2114766af2c638add98503a: Bootstrap starting.
I20260811 09:08:18.516789 7137 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d95023fbc2114766af2c638add98503a: Neither blocks nor log segments found. Creating new log.
I20260811 09:08:18.518030 7137 log.cc:824] T 00000000000000000000000000000000 P d95023fbc2114766af2c638add98503a: Log is configured to *not* fsync() on all Append() calls
I20260811 09:08:18.521659 7137 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d95023fbc2114766af2c638add98503a: No bootstrap required, opened a new log
I20260811 09:08:18.524008 7143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:18.510756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47600 (local address 127.28.154.190:46121)
0811 09:08:18.511936 (+ 1180us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:18.511948 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:18.511997 (+ 49us) server_negotiation.cc:407] Connection header received
0811 09:08:18.514001 (+ 2004us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:18.514031 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:18.514522 (+ 491us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:18.514909 (+ 387us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:18.515483 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:18.516834 (+ 1351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:18.518488 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:18.519610 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:18.519806 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:18.521019 (+ 1213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:18.521028 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:18.521044 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:08:18.521622 (+ 578us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:18.521710 (+ 88us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:08:18.521732 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:18.521894 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:18.522139 (+ 245us) server_negotiation.cc:299] Negotiation successful
0811 09:08:18.522927 (+ 788us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":616,"thread_start_us":366,"threads_started":1}
I20260811 09:08:18.528303 29290 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 09:08:18.530426 7137 raft_consensus.cc:348] T 00000000000000000000000000000000 P d95023fbc2114766af2c638add98503a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d95023fbc2114766af2c638add98503a" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 46121 } }
I20260811 09:08:18.530901 7137 raft_consensus.cc:374] T 00000000000000000000000000000000 P d95023fbc2114766af2c638add98503a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:08:18.531008 7137 raft_consensus.cc:729] T 00000000000000000000000000000000 P d95023fbc2114766af2c638add98503a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d95023fbc2114766af2c638add98503a, State: Initialized, Role: FOLLOWER
I20260811 09:08:18.531672 7137 consensus_queue.cc:260] T 00000000000000000000000000000000 P d95023fbc2114766af2c638add98503a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d95023fbc2114766af2c638add98503a" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 46121 } }
I20260811 09:08:18.531908 7137 raft_consensus.cc:388] T 00000000000000000000000000000000 P d95023fbc2114766af2c638add98503a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:08:18.532042 7137 raft_consensus.cc:482] T 00000000000000000000000000000000 P d95023fbc2114766af2c638add98503a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:08:18.532203 7137 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d95023fbc2114766af2c638add98503a [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:08:18.534456 7137 raft_consensus.cc:504] T 00000000000000000000000000000000 P d95023fbc2114766af2c638add98503a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d95023fbc2114766af2c638add98503a" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 46121 } }
I20260811 09:08:18.534981 7137 leader_election.cc:302] T 00000000000000000000000000000000 P d95023fbc2114766af2c638add98503a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d95023fbc2114766af2c638add98503a; no voters:
I20260811 09:08:18.535631 7137 leader_election.cc:288] T 00000000000000000000000000000000 P d95023fbc2114766af2c638add98503a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:08:18.535770 7145 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d95023fbc2114766af2c638add98503a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:08:18.536868 7145 raft_consensus.cc:686] T 00000000000000000000000000000000 P d95023fbc2114766af2c638add98503a [term 1 LEADER]: Becoming Leader. State: Replica: d95023fbc2114766af2c638add98503a, State: Running, Role: LEADER
I20260811 09:08:18.537402 7145 consensus_queue.cc:237] T 00000000000000000000000000000000 P d95023fbc2114766af2c638add98503a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d95023fbc2114766af2c638add98503a" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 46121 } }
I20260811 09:08:18.537971 7137 sys_catalog.cc:564] T 00000000000000000000000000000000 P d95023fbc2114766af2c638add98503a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:08:18.540210 7146 sys_catalog.cc:455] T 00000000000000000000000000000000 P d95023fbc2114766af2c638add98503a [sys.catalog]: SysCatalogTable state changed. Reason: New leader d95023fbc2114766af2c638add98503a. Latest consensus state: current_term: 1 leader_uuid: "d95023fbc2114766af2c638add98503a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d95023fbc2114766af2c638add98503a" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 46121 } } }
I20260811 09:08:18.540671 7146 sys_catalog.cc:458] T 00000000000000000000000000000000 P d95023fbc2114766af2c638add98503a [sys.catalog]: This master's current role is: LEADER
I20260811 09:08:18.541792 7147 sys_catalog.cc:455] T 00000000000000000000000000000000 P d95023fbc2114766af2c638add98503a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d95023fbc2114766af2c638add98503a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d95023fbc2114766af2c638add98503a" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 46121 } } }
I20260811 09:08:18.542148 7147 sys_catalog.cc:458] T 00000000000000000000000000000000 P d95023fbc2114766af2c638add98503a [sys.catalog]: This master's current role is: LEADER
I20260811 09:08:18.543941 7160 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:08:18.548808 7160 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:08:18.557134 7160 catalog_manager.cc:1347] Generated new cluster ID: 7b0d0be973ba4632875e425d033d3bbf
I20260811 09:08:18.557263 7160 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:08:18.571810 7160 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:08:18.573734 7160 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:08:18.581578 7160 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d95023fbc2114766af2c638add98503a: Generated new TSK 0
I20260811 09:08:18.582574 7160 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 09:08:18.610020 7143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:18.601872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47610 (local address 127.28.154.190:46121)
0811 09:08:18.602100 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:18.602108 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:18.602592 (+ 484us) server_negotiation.cc:407] Connection header received
0811 09:08:18.602880 (+ 288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:18.602884 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:18.602974 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:18.603120 (+ 146us) server_negotiation.cc:226] Negotiated authn=JWT
0811 09:08:18.603684 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:18.604652 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:18.606143 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:18.606605 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:18.606733 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:18.607253 (+ 520us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 09:08:18.609214 (+ 1961us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42279/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0811 09:08:18.609531 (+ 317us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:47610: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42279/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":68}
W20260811 09:08:18.610167 7183 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:08:18.601789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:46121 (local address 127.0.0.1:47610)
0811 09:08:18.602440 (+ 651us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:18.602461 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:18.602669 (+ 208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:18.603159 (+ 490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:18.603164 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:18.603177 (+ 13us) client_negotiation.cc:189] Negotiated authn=JWT
0811 09:08:18.603523 (+ 346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:18.603537 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:18.604847 (+ 1310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:18.604853 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:18.605991 (+ 1138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:18.606004 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:18.606803 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:18.606811 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:18.606952 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:18.606986 (+ 34us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 09:08:18.609404 (+ 2418us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42279/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0811 09:08:18.609654 (+ 250us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.28.154.190:46121: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42279/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":321,"threads_started":1}
W20260811 09:08:18.610301 7143 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:47610: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42279/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
I20260811 09:08:18.611202 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 7006
I20260811 09:08:18.612439 7117 generic_service.cc:196] Checking for leaks (request via RPC)
E20260811 09:08:18.622310 7184 webserver.cc:533] Webserver: sslize failed:
E20260811 09:08:18.622435 7184 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 09:08:18.704305 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 7006
2026-08-11T09:08:18Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (744 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-11T09:08:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:08:18Z Disabled control of system clock
I20260811 09:08:18.796279 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786439250192320-29290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786439250192320-29290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786439250192320-29290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.154.190:36329
--webserver_interface=127.28.154.190
--webserver_port=0
--builtin_ntp_servers=127.28.154.148:39991
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.154.190:36329
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786439250192320-29290-0/testchainca.cert with env {}
W20260811 09:08:19.137418 7191 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:08:19.138101 7191 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:08:19.138222 7191 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:08:19.145772 7191 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:08:19.145895 7191 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:08:19.145933 7191 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:08:19.145990 7191 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:08:19.146039 7191 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 09:08:19.146100 7191 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 09:08:19.146140 7191 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260811 09:08:19.157068 7191 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:39991
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786439250192320-29290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786439250192320-29290-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.154.190:36329
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.154.190:36329
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.154.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786439250192320-29290-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786439250192320-29290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:08:19.158547 7191 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:08:19.159873 7191 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:08:19.171633 7201 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:08:19.171591 7206 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:08:19.171605 7204 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:08:19.172662 7191 server_base.cc:1034] running on GCE node
I20260811 09:08:19.173452 7191 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:08:19.174887 7191 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:08:19.176091 7191 hybrid_clock.cc:648] HybridClock initialized: now 1786439299176036 us; error 49 us; skew 500 ppm
I20260811 09:08:19.176503 7191 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 09:08:19.182684 7191 webserver.cc:460] Webserver started at http://127.28.154.190:44197/ using document root <none> and password file <none>
I20260811 09:08:19.183462 7191 fs_manager.cc:359] Metadata directory not provided
I20260811 09:08:19.183562 7191 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:08:19.183892 7191 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:08:19.186965 7191 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786439250192320-29290-0/minicluster-data/master-0/data/instance:
uuid: "ee6d28d0cc4f46f0aa970828cd2e4e9d"
format_stamp: "Formatted at 2026-08-11 09:08:19 on dist-test-slave-xt4k"
server_key: "2c1d9ea36b433b18e9bd78adb0a554d2"
server_key_iv: "f67834dd7afe1a49016440564893ec0c"
server_key_version: "encryptionkey@0"
I20260811 09:08:19.187759 7191 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786439250192320-29290-0/minicluster-data/master-0/wal/instance:
uuid: "ee6d28d0cc4f46f0aa970828cd2e4e9d"
format_stamp: "Formatted at 2026-08-11 09:08:19 on dist-test-slave-xt4k"
server_key: "2c1d9ea36b433b18e9bd78adb0a554d2"
server_key_iv: "f67834dd7afe1a49016440564893ec0c"
server_key_version: "encryptionkey@0"
I20260811 09:08:19.193136 7191 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 09:08:19.196820 7217 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:08:19.198314 7191 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260811 09:08:19.198532 7191 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786439250192320-29290-0/minicluster-data/master-0/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786439250192320-29290-0/minicluster-data/master-0/wal
uuid: "ee6d28d0cc4f46f0aa970828cd2e4e9d"
format_stamp: "Formatted at 2026-08-11 09:08:19 on dist-test-slave-xt4k"
server_key: "2c1d9ea36b433b18e9bd78adb0a554d2"
server_key_iv: "f67834dd7afe1a49016440564893ec0c"
server_key_version: "encryptionkey@0"
I20260811 09:08:19.198751 7191 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786439250192320-29290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786439250192320-29290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786439250192320-29290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:08:19.221459 7191 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:08:19.222416 7191 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:08:19.222710 7191 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:08:19.240636 7191 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.190:36329
I20260811 09:08:19.240660 7309 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.190:36329 every 8 connection(s)
I20260811 09:08:19.242097 7191 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
I20260811 09:08:19.242867 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 7191
I20260811 09:08:19.243098 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786439250192320-29290-0/minicluster-data/master-0/wal/instance
I20260811 09:08:19.243789 29290 external_mini_cluster.cc:1428] Setting key 0637b48941691132c39752879a8f7ef8
I20260811 09:08:19.246876 7311 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:08:19.257900 7321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:19.245718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59606 (local address 127.28.154.190:36329)
0811 09:08:19.246739 (+ 1021us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:19.246751 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:19.246796 (+ 45us) server_negotiation.cc:407] Connection header received
0811 09:08:19.248787 (+ 1991us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:19.248812 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:19.249333 (+ 521us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:19.249750 (+ 417us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:19.250347 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:19.251825 (+ 1478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:19.253187 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:19.253919 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:19.254090 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:19.255301 (+ 1211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:19.255309 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:19.255325 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:08:19.255931 (+ 606us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:19.256012 (+ 81us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:08:19.256038 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:19.256149 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:19.256413 (+ 264us) server_negotiation.cc:299] Negotiation successful
0811 09:08:19.256874 (+ 461us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":545,"thread_start_us":336,"threads_started":1}
I20260811 09:08:19.259341 7311 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee6d28d0cc4f46f0aa970828cd2e4e9d: Bootstrap starting.
I20260811 09:08:19.262223 29290 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 09:08:19.262934 7311 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ee6d28d0cc4f46f0aa970828cd2e4e9d: Neither blocks nor log segments found. Creating new log.
I20260811 09:08:19.263960 7311 log.cc:824] T 00000000000000000000000000000000 P ee6d28d0cc4f46f0aa970828cd2e4e9d: Log is configured to *not* fsync() on all Append() calls
I20260811 09:08:19.267231 7311 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee6d28d0cc4f46f0aa970828cd2e4e9d: No bootstrap required, opened a new log
I20260811 09:08:19.275542 7311 raft_consensus.cc:348] T 00000000000000000000000000000000 P ee6d28d0cc4f46f0aa970828cd2e4e9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee6d28d0cc4f46f0aa970828cd2e4e9d" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 36329 } }
I20260811 09:08:19.276098 7311 raft_consensus.cc:374] T 00000000000000000000000000000000 P ee6d28d0cc4f46f0aa970828cd2e4e9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:08:19.276193 7311 raft_consensus.cc:729] T 00000000000000000000000000000000 P ee6d28d0cc4f46f0aa970828cd2e4e9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee6d28d0cc4f46f0aa970828cd2e4e9d, State: Initialized, Role: FOLLOWER
I20260811 09:08:19.276794 7311 consensus_queue.cc:260] T 00000000000000000000000000000000 P ee6d28d0cc4f46f0aa970828cd2e4e9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee6d28d0cc4f46f0aa970828cd2e4e9d" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 36329 } }
I20260811 09:08:19.277012 7311 raft_consensus.cc:388] T 00000000000000000000000000000000 P ee6d28d0cc4f46f0aa970828cd2e4e9d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:08:19.277117 7311 raft_consensus.cc:482] T 00000000000000000000000000000000 P ee6d28d0cc4f46f0aa970828cd2e4e9d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:08:19.277256 7311 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ee6d28d0cc4f46f0aa970828cd2e4e9d [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:08:19.279851 7311 raft_consensus.cc:504] T 00000000000000000000000000000000 P ee6d28d0cc4f46f0aa970828cd2e4e9d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee6d28d0cc4f46f0aa970828cd2e4e9d" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 36329 } }
I20260811 09:08:19.280459 7311 leader_election.cc:302] T 00000000000000000000000000000000 P ee6d28d0cc4f46f0aa970828cd2e4e9d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ee6d28d0cc4f46f0aa970828cd2e4e9d; no voters:
I20260811 09:08:19.281068 7311 leader_election.cc:288] T 00000000000000000000000000000000 P ee6d28d0cc4f46f0aa970828cd2e4e9d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:08:19.281208 7326 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ee6d28d0cc4f46f0aa970828cd2e4e9d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:08:19.281911 7326 raft_consensus.cc:686] T 00000000000000000000000000000000 P ee6d28d0cc4f46f0aa970828cd2e4e9d [term 1 LEADER]: Becoming Leader. State: Replica: ee6d28d0cc4f46f0aa970828cd2e4e9d, State: Running, Role: LEADER
I20260811 09:08:19.282527 7326 consensus_queue.cc:237] T 00000000000000000000000000000000 P ee6d28d0cc4f46f0aa970828cd2e4e9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee6d28d0cc4f46f0aa970828cd2e4e9d" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 36329 } }
I20260811 09:08:19.283241 7311 sys_catalog.cc:564] T 00000000000000000000000000000000 P ee6d28d0cc4f46f0aa970828cd2e4e9d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:08:19.285321 7328 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee6d28d0cc4f46f0aa970828cd2e4e9d [sys.catalog]: SysCatalogTable state changed. Reason: New leader ee6d28d0cc4f46f0aa970828cd2e4e9d. Latest consensus state: current_term: 1 leader_uuid: "ee6d28d0cc4f46f0aa970828cd2e4e9d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee6d28d0cc4f46f0aa970828cd2e4e9d" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 36329 } } }
I20260811 09:08:19.285952 7328 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee6d28d0cc4f46f0aa970828cd2e4e9d [sys.catalog]: This master's current role is: LEADER
I20260811 09:08:19.285495 7327 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee6d28d0cc4f46f0aa970828cd2e4e9d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ee6d28d0cc4f46f0aa970828cd2e4e9d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee6d28d0cc4f46f0aa970828cd2e4e9d" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 36329 } } }
I20260811 09:08:19.286079 7327 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee6d28d0cc4f46f0aa970828cd2e4e9d [sys.catalog]: This master's current role is: LEADER
I20260811 09:08:19.288065 7336 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:08:19.293720 7336 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:08:19.305459 7336 catalog_manager.cc:1347] Generated new cluster ID: f15564c57707477195f5093362410c1a
I20260811 09:08:19.305581 7336 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:08:19.318331 7336 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:08:19.319712 7336 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:08:19.329980 7336 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ee6d28d0cc4f46f0aa970828cd2e4e9d: Generated new TSK 0
I20260811 09:08:19.330969 7336 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 09:08:20.208575 7321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:19.343575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59612 (local address 127.28.154.190:36329)
0811 09:08:19.343772 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:19.343778 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:19.344085 (+ 307us) server_negotiation.cc:407] Connection header received
0811 09:08:19.344299 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:19.344305 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:19.344426 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:19.344614 (+ 188us) server_negotiation.cc:226] Negotiated authn=JWT
0811 09:08:19.345064 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:19.346203 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:19.347623 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:19.348316 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:19.348469 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:19.348989 (+ 520us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 09:08:20.207874 (+858885us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0811 09:08:20.208227 (+ 353us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:59612: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":71}
W20260811 09:08:20.208724 7321 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:59612: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260811 09:08:20.208911 7369 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:08:19.343402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:36329 (local address 127.0.0.1:59612)
0811 09:08:19.343982 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:19.343998 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:19.344124 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:19.344632 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:19.344635 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:19.344646 (+ 11us) client_negotiation.cc:189] Negotiated authn=JWT
0811 09:08:19.344913 (+ 267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:19.344922 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:19.346363 (+ 1441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:19.346367 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:19.347473 (+ 1106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:19.347486 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:19.348499 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:19.348503 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:19.348654 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:19.348691 (+ 37us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 09:08:20.208186 (+859495us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0811 09:08:20.208542 (+ 356us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.28.154.190:36329: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":298,"threads_started":1}
I20260811 09:08:20.209944 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 7191
I20260811 09:08:20.211097 7302 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:08:20.312690 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 7191
2026-08-11T09:08:20Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (1593 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (425 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (439 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439250192320-29290-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:08:21 dist-test-slave-xt4k krb5kdc[7395](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:08:21 dist-test-slave-xt4k krb5kdc[7395](info): set up 2 sockets
Aug 11 09:08:21 dist-test-slave-xt4k krb5kdc[7395](info): commencing operation
krb5kdc: starting...
W20260811 09:08:22.456328 29290 mini_kdc.cc:121] Time spent starting KDC: real 1.234s user 0.000s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439250192320-29290-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439250192320-29290-0/krb5kdc/test-user.keytab.
Aug 11 09:08:22 dist-test-slave-xt4k krb5kdc[7395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439302, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:08:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:08:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.154.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.154.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.154.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab.
Entry for principal kudu/127.28.154.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab.
I20260811 09:08:22.703627 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439250192320-29290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439250192320-29290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439250192320-29290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.154.190:39219
--webserver_interface=127.28.154.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab
--principal=kudu/127.28.154.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:35053
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.154.190:39219
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:08:23.045884 7422 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:08:23.046205 7422 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:08:23.046298 7422 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:08:23.054160 7422 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:08:23.054301 7422 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:08:23.054349 7422 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:08:23.054407 7422 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:08:23.054451 7422 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:08:23.065375 7422 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:35053
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439250192320-29290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439250192320-29290-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.154.190:39219
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab
--principal=kudu/127.28.154.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.154.190:39219
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.154.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439250192320-29290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:08:23.066757 7422 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:08:23.068153 7422 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:08:23.083734 7438 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:08:23.083840 7431 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:08:23.084059 7428 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:08:23.084995 7422 server_base.cc:1034] running on GCE node
I20260811 09:08:23.086020 7422 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:08:23.087705 7422 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:08:23.088960 7422 hybrid_clock.cc:648] HybridClock initialized: now 1786439303088913 us; error 35 us; skew 500 ppm
I20260811 09:08:23.089409 7422 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:08:23 dist-test-slave-xt4k krb5kdc[7395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439303, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:08:23.093466 7422 init.cc:377] Logged in from keytab as kudu/127.28.154.190@KRBTEST.COM (short username kudu)
I20260811 09:08:23.095383 7422 webserver.cc:460] Webserver started at http://127.28.154.190:40881/ using document root <none> and password file <none>
I20260811 09:08:23.096316 7422 fs_manager.cc:359] Metadata directory not provided
I20260811 09:08:23.096510 7422 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:08:23.096906 7422 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:08:23.100571 7422 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439250192320-29290-0/minicluster-data/master-0/data/instance:
uuid: "0880ea21bd6a46268083e669760be729"
format_stamp: "Formatted at 2026-08-11 09:08:23 on dist-test-slave-xt4k"
server_key: "e1d38c49d008188a77a64b78228ac9a8"
server_key_iv: "75013ea378f4f4ce9da5e15e44e7cbbd"
server_key_version: "encryptionkey@0"
I20260811 09:08:23.101454 7422 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439250192320-29290-0/minicluster-data/master-0/wal/instance:
uuid: "0880ea21bd6a46268083e669760be729"
format_stamp: "Formatted at 2026-08-11 09:08:23 on dist-test-slave-xt4k"
server_key: "e1d38c49d008188a77a64b78228ac9a8"
server_key_iv: "75013ea378f4f4ce9da5e15e44e7cbbd"
server_key_version: "encryptionkey@0"
I20260811 09:08:23.107553 7422 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.002s
I20260811 09:08:23.111610 7454 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:08:23.113348 7422 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 09:08:23.113545 7422 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439250192320-29290-0/minicluster-data/master-0/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439250192320-29290-0/minicluster-data/master-0/wal
uuid: "0880ea21bd6a46268083e669760be729"
format_stamp: "Formatted at 2026-08-11 09:08:23 on dist-test-slave-xt4k"
server_key: "e1d38c49d008188a77a64b78228ac9a8"
server_key_iv: "75013ea378f4f4ce9da5e15e44e7cbbd"
server_key_version: "encryptionkey@0"
I20260811 09:08:23.113754 7422 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439250192320-29290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439250192320-29290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439250192320-29290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:08:23.146848 7422 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:08:23.152331 7422 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:08:23.152699 7422 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:08:23.171295 7422 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.190:39219
I20260811 09:08:23.171331 7554 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.190:39219 every 8 connection(s)
I20260811 09:08:23.172746 7422 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
I20260811 09:08:23.176934 7555 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:08:23.179210 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 7422
I20260811 09:08:23.179476 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439250192320-29290-0/minicluster-data/master-0/wal/instance
I20260811 09:08:23.180482 29290 external_mini_cluster.cc:1428] Setting key cbf9a663fa2232a05d8c615208a0e382
I20260811 09:08:23.190883 7555 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0880ea21bd6a46268083e669760be729: Bootstrap starting.
I20260811 09:08:23.195088 7555 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0880ea21bd6a46268083e669760be729: Neither blocks nor log segments found. Creating new log.
I20260811 09:08:23.196416 7555 log.cc:824] T 00000000000000000000000000000000 P 0880ea21bd6a46268083e669760be729: Log is configured to *not* fsync() on all Append() calls
Aug 11 09:08:23 dist-test-slave-xt4k krb5kdc[7395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439302, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.154.190@KRBTEST.COM
I20260811 09:08:23.199437 7555 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0880ea21bd6a46268083e669760be729: No bootstrap required, opened a new log
I20260811 09:08:23.205199 7558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:23.182989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32916 (local address 127.28.154.190:39219)
0811 09:08:23.183868 (+ 879us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:23.183881 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:23.183926 (+ 45us) server_negotiation.cc:407] Connection header received
0811 09:08:23.185731 (+ 1805us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:23.185759 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:23.186187 (+ 428us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:23.186552 (+ 365us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:23.188507 (+ 1955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:23.190078 (+ 1571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:23.191795 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:23.192595 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:23.192775 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:23.197576 (+ 4801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:23.197587 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:23.197604 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:08:23.197647 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:23.200920 (+ 3273us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:23.201662 (+ 742us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:23.201670 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:23.201677 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:23.201853 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:23.202241 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:23.202248 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:23.202250 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:23.202857 (+ 607us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:23.203162 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:23.203609 (+ 447us) server_negotiation.cc:299] Negotiation successful
0811 09:08:23.204066 (+ 457us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":639,"thread_start_us":390,"threads_started":1}
I20260811 09:08:23.207841 7555 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0880ea21bd6a46268083e669760be729 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0880ea21bd6a46268083e669760be729" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 39219 } }
I20260811 09:08:23.208273 7555 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0880ea21bd6a46268083e669760be729 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:08:23.208451 7555 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0880ea21bd6a46268083e669760be729 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0880ea21bd6a46268083e669760be729, State: Initialized, Role: FOLLOWER
I20260811 09:08:23.209002 7555 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0880ea21bd6a46268083e669760be729 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0880ea21bd6a46268083e669760be729" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 39219 } }
I20260811 09:08:23.209220 7555 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0880ea21bd6a46268083e669760be729 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:08:23.209319 7555 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0880ea21bd6a46268083e669760be729 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:08:23.209435 7555 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0880ea21bd6a46268083e669760be729 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:08:23.211692 7555 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0880ea21bd6a46268083e669760be729 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0880ea21bd6a46268083e669760be729" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 39219 } }
I20260811 09:08:23.212281 7555 leader_election.cc:302] T 00000000000000000000000000000000 P 0880ea21bd6a46268083e669760be729 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0880ea21bd6a46268083e669760be729; no voters:
I20260811 09:08:23.212966 7555 leader_election.cc:288] T 00000000000000000000000000000000 P 0880ea21bd6a46268083e669760be729 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:08:23.213050 7566 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0880ea21bd6a46268083e669760be729 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:08:23.213835 7566 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0880ea21bd6a46268083e669760be729 [term 1 LEADER]: Becoming Leader. State: Replica: 0880ea21bd6a46268083e669760be729, State: Running, Role: LEADER
I20260811 09:08:23.214342 7566 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0880ea21bd6a46268083e669760be729 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0880ea21bd6a46268083e669760be729" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 39219 } }
I20260811 09:08:23.215044 7555 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0880ea21bd6a46268083e669760be729 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:08:23.217633 7569 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0880ea21bd6a46268083e669760be729 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0880ea21bd6a46268083e669760be729. Latest consensus state: current_term: 1 leader_uuid: "0880ea21bd6a46268083e669760be729" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0880ea21bd6a46268083e669760be729" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 39219 } } }
I20260811 09:08:23.218017 7569 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0880ea21bd6a46268083e669760be729 [sys.catalog]: This master's current role is: LEADER
I20260811 09:08:23.218698 7568 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0880ea21bd6a46268083e669760be729 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0880ea21bd6a46268083e669760be729" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0880ea21bd6a46268083e669760be729" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 39219 } } }
I20260811 09:08:23.219002 7568 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0880ea21bd6a46268083e669760be729 [sys.catalog]: This master's current role is: LEADER
I20260811 09:08:23.220090 7575 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:08:23.225441 7575 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:08:23.243119 7575 catalog_manager.cc:1347] Generated new cluster ID: d11eee564d644905868881ff23819a9a
I20260811 09:08:23.243263 7575 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:08:23.268792 7575 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:08:23.270217 7575 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:08:23.279222 7575 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0880ea21bd6a46268083e669760be729: Generated new TSK 0
I20260811 09:08:23.280081 7575 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.28.154.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.154.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.154.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.129.keytab.
Entry for principal kudu/127.28.154.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.129.keytab.
I20260811 09:08:23.358623 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439250192320-29290-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439250192320-29290-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439250192320-29290-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439250192320-29290-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.154.129:0
--local_ip_for_outbound_sockets=127.28.154.129
--webserver_interface=127.28.154.129
--webserver_port=0
--tserver_master_addrs=127.28.154.190:39219
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.129.keytab
--principal=kudu/127.28.154.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:35053
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:08:23.724433 7601 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:08:23.724752 7601 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:08:23.724850 7601 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:08:23.732826 7601 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:08:23.732967 7601 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:08:23.733089 7601 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.154.129
I20260811 09:08:23.743728 7601 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:35053
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439250192320-29290-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439250192320-29290-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.129.keytab
--principal=kudu/127.28.154.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.154.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439250192320-29290-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.154.129
--webserver_port=0
--tserver_master_addrs=127.28.154.190:39219
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.154.129
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439250192320-29290-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:08:23.745154 7601 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:08:23.746428 7601 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:08:23.759642 7608 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:08:23.759611 7609 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:08:23.759577 7613 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:08:23.760653 7601 server_base.cc:1034] running on GCE node
I20260811 09:08:23.761284 7601 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:08:23.762270 7601 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:08:23.763494 7601 hybrid_clock.cc:648] HybridClock initialized: now 1786439303763434 us; error 52 us; skew 500 ppm
I20260811 09:08:23.763854 7601 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:08:23 dist-test-slave-xt4k krb5kdc[7395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439303, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:08:23.767023 7601 init.cc:377] Logged in from keytab as kudu/127.28.154.129@KRBTEST.COM (short username kudu)
I20260811 09:08:23.768548 7601 webserver.cc:460] Webserver started at http://127.28.154.129:38533/ using document root <none> and password file <none>
I20260811 09:08:23.769212 7601 fs_manager.cc:359] Metadata directory not provided
I20260811 09:08:23.769316 7601 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:08:23.769596 7601 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:08:23.772636 7601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439250192320-29290-0/minicluster-data/ts-0/data/instance:
uuid: "b359b8a2b8424d8cadcc59c4d254e7d7"
format_stamp: "Formatted at 2026-08-11 09:08:23 on dist-test-slave-xt4k"
server_key: "e8d107ce66fd117d808e78b685df4d83"
server_key_iv: "073ae4efef82546056e766e6532d29fb"
server_key_version: "encryptionkey@0"
I20260811 09:08:23.773376 7601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439250192320-29290-0/minicluster-data/ts-0/wal/instance:
uuid: "b359b8a2b8424d8cadcc59c4d254e7d7"
format_stamp: "Formatted at 2026-08-11 09:08:23 on dist-test-slave-xt4k"
server_key: "e8d107ce66fd117d808e78b685df4d83"
server_key_iv: "073ae4efef82546056e766e6532d29fb"
server_key_version: "encryptionkey@0"
I20260811 09:08:23.778680 7601 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 09:08:23.782348 7625 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:08:23.783826 7601 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 09:08:23.783995 7601 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439250192320-29290-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439250192320-29290-0/minicluster-data/ts-0/wal
uuid: "b359b8a2b8424d8cadcc59c4d254e7d7"
format_stamp: "Formatted at 2026-08-11 09:08:23 on dist-test-slave-xt4k"
server_key: "e8d107ce66fd117d808e78b685df4d83"
server_key_iv: "073ae4efef82546056e766e6532d29fb"
server_key_version: "encryptionkey@0"
I20260811 09:08:23.784188 7601 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439250192320-29290-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439250192320-29290-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439250192320-29290-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:08:23.802583 7601 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:08:23.806948 7601 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:08:23.807224 7601 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:08:23.808245 7601 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:08:23.809662 7601 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:08:23.809743 7601 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:08:23.809870 7601 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:08:23.809923 7601 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:08:23.841291 7601 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.129:38515
I20260811 09:08:23.841331 7836 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.129:38515 every 8 connection(s)
I20260811 09:08:23.842587 7601 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439250192320-29290-0/minicluster-data/ts-0/data/info.pb
I20260811 09:08:23.849017 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 7601
I20260811 09:08:23.849242 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439250192320-29290-0/minicluster-data/ts-0/wal/instance
I20260811 09:08:23.849867 29290 external_mini_cluster.cc:1428] Setting key c2fb2de44cd73b57aaa4529caff567a9
Aug 11 09:08:23 dist-test-slave-xt4k krb5kdc[7395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439303, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.129@KRBTEST.COM for kudu/127.28.154.190@KRBTEST.COM
I20260811 09:08:23.863629 7842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:23.845221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.129:59323 (local address 127.28.154.190:39219)
0811 09:08:23.845712 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:23.845718 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:23.846864 (+ 1146us) server_negotiation.cc:407] Connection header received
0811 09:08:23.848784 (+ 1920us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:23.848791 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:23.848877 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:23.849026 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:23.850696 (+ 1670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:23.851696 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:23.853327 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:23.854196 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:23.854343 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:23.857991 (+ 3648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:23.857996 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:23.858002 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:08:23.858038 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:23.860532 (+ 2494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:23.861280 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:23.861284 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:23.861287 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:23.861410 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:23.861839 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:23.861844 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:23.861846 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:23.862118 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:23.862253 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:23.863018 (+ 765us) server_negotiation.cc:299] Negotiation successful
0811 09:08:23.863255 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":386,"thread_start_us":285,"threads_started":1}
I20260811 09:08:23.864655 7843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:23.845783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:39219 (local address 127.28.154.129:59323)
0811 09:08:23.846625 (+ 842us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:23.846663 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:23.848438 (+ 1775us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:23.849282 (+ 844us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:23.849291 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:23.849894 (+ 603us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:08:23.850510 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:23.850530 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:23.851872 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:23.851876 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:23.853173 (+ 1297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:23.853188 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:23.854469 (+ 1281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:23.854478 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:23.854717 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:23.855369 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:08:23.855399 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:08:23.857757 (+ 2358us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:08:23.860679 (+ 2922us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:08:23.860690 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:08:23.860709 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:08:23.861149 (+ 440us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:08:23.861515 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:08:23.861521 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:08:23.861525 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:08:23.861721 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:08:23.862244 (+ 523us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:08:23.862253 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:08:23.862625 (+ 372us) client_negotiation.cc:769] Sending connection context
0811 09:08:23.862885 (+ 260us) client_negotiation.cc:240] Negotiation successful
0811 09:08:23.863441 (+ 556us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":580,"thread_start_us":300,"threads_started":1}
I20260811 09:08:23.866276 7838 heartbeater.cc:347] Connected to a master server at 127.28.154.190:39219
I20260811 09:08:23.866671 7838 heartbeater.cc:464] Registering TS with master...
I20260811 09:08:23.867568 7838 heartbeater.cc:511] Master 127.28.154.190:39219 requested a full tablet report, sending...
I20260811 09:08:23.870194 7493 ts_manager.cc:194] Registered new tserver with Master: b359b8a2b8424d8cadcc59c4d254e7d7 (127.28.154.129:38515)
I20260811 09:08:23.872277 7493 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.154.129@KRBTEST.COM'} at 127.28.154.129:59323
WARNING: no policy specified for kudu/127.28.154.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.154.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.154.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.130.keytab.
Entry for principal kudu/127.28.154.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.130.keytab.
I20260811 09:08:23.907882 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439250192320-29290-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439250192320-29290-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439250192320-29290-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439250192320-29290-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.154.130:0
--local_ip_for_outbound_sockets=127.28.154.130
--webserver_interface=127.28.154.130
--webserver_port=0
--tserver_master_addrs=127.28.154.190:39219
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.130.keytab
--principal=kudu/127.28.154.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:35053
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:08:24.264053 7850 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:08:24.264379 7850 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:08:24.264523 7850 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:08:24.271719 7850 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:08:24.271843 7850 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:08:24.272006 7850 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.154.130
I20260811 09:08:24.282285 7850 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:35053
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439250192320-29290-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439250192320-29290-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.130.keytab
--principal=kudu/127.28.154.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.154.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439250192320-29290-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.154.130
--webserver_port=0
--tserver_master_addrs=127.28.154.190:39219
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.154.130
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439250192320-29290-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:08:24.283681 7850 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:08:24.285259 7850 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:08:24.298681 7864 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:08:24.298720 7861 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:08:24.298879 7858 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:08:24.299644 7850 server_base.cc:1034] running on GCE node
I20260811 09:08:24.300271 7850 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:08:24.301302 7850 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:08:24.302547 7850 hybrid_clock.cc:648] HybridClock initialized: now 1786439304302474 us; error 51 us; skew 500 ppm
I20260811 09:08:24.302898 7850 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:08:24 dist-test-slave-xt4k krb5kdc[7395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439304, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:08:24.308904 7850 init.cc:377] Logged in from keytab as kudu/127.28.154.130@KRBTEST.COM (short username kudu)
I20260811 09:08:24.310482 7850 webserver.cc:460] Webserver started at http://127.28.154.130:39549/ using document root <none> and password file <none>
I20260811 09:08:24.311205 7850 fs_manager.cc:359] Metadata directory not provided
I20260811 09:08:24.311331 7850 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:08:24.311643 7850 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:08:24.315044 7850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439250192320-29290-0/minicluster-data/ts-1/data/instance:
uuid: "96487104291043f2bf82fa52fe7a1e33"
format_stamp: "Formatted at 2026-08-11 09:08:24 on dist-test-slave-xt4k"
server_key: "d48528346ba93487e522d108eb2c337c"
server_key_iv: "030f3d10739559f71b7ef945ec1357c7"
server_key_version: "encryptionkey@0"
I20260811 09:08:24.315903 7850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439250192320-29290-0/minicluster-data/ts-1/wal/instance:
uuid: "96487104291043f2bf82fa52fe7a1e33"
format_stamp: "Formatted at 2026-08-11 09:08:24 on dist-test-slave-xt4k"
server_key: "d48528346ba93487e522d108eb2c337c"
server_key_iv: "030f3d10739559f71b7ef945ec1357c7"
server_key_version: "encryptionkey@0"
I20260811 09:08:24.321413 7850 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.003s
I20260811 09:08:24.325189 7884 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:08:24.326635 7850 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 09:08:24.326818 7850 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439250192320-29290-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439250192320-29290-0/minicluster-data/ts-1/wal
uuid: "96487104291043f2bf82fa52fe7a1e33"
format_stamp: "Formatted at 2026-08-11 09:08:24 on dist-test-slave-xt4k"
server_key: "d48528346ba93487e522d108eb2c337c"
server_key_iv: "030f3d10739559f71b7ef945ec1357c7"
server_key_version: "encryptionkey@0"
I20260811 09:08:24.327035 7850 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439250192320-29290-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439250192320-29290-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439250192320-29290-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:08:24.344625 7850 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:08:24.348747 7850 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:08:24.349018 7850 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:08:24.350004 7850 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:08:24.351356 7850 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:08:24.351445 7850 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:08:24.351588 7850 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:08:24.351641 7850 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:08:24.383594 7850 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.130:46459
I20260811 09:08:24.383647 8101 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.130:46459 every 8 connection(s)
I20260811 09:08:24.384883 7850 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439250192320-29290-0/minicluster-data/ts-1/data/info.pb
I20260811 09:08:24.393533 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 7850
I20260811 09:08:24.393715 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439250192320-29290-0/minicluster-data/ts-1/wal/instance
I20260811 09:08:24.394152 29290 external_mini_cluster.cc:1428] Setting key feaf021e41831eadcf08fb22c1061956
Aug 11 09:08:24 dist-test-slave-xt4k krb5kdc[7395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439304, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.130@KRBTEST.COM for kudu/127.28.154.190@KRBTEST.COM
I20260811 09:08:24.406345 8106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:24.387445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.130:59377 (local address 127.28.154.190:39219)
0811 09:08:24.387898 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:24.387903 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:24.388880 (+ 977us) server_negotiation.cc:407] Connection header received
0811 09:08:24.390759 (+ 1879us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:24.390765 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:24.390847 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:24.390946 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:24.392521 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:24.393327 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:24.395183 (+ 1856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:24.395718 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:24.395818 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:24.399605 (+ 3787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:24.399611 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:24.399616 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:08:24.399657 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:24.402894 (+ 3237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:24.403729 (+ 835us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:24.403735 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:24.403737 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:24.403896 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:24.404421 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:24.404425 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:24.404428 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:24.404760 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:24.404907 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:24.405725 (+ 818us) server_negotiation.cc:299] Negotiation successful
0811 09:08:24.405975 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":245,"threads_started":1}
I20260811 09:08:24.407232 8107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:24.387861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:39219 (local address 127.28.154.130:59377)
0811 09:08:24.388698 (+ 837us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:24.388743 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:24.390475 (+ 1732us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:24.391205 (+ 730us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:24.391214 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:24.391729 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:08:24.392307 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:24.392325 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:24.393513 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:24.393517 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:24.395031 (+ 1514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:24.395055 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:24.395955 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:24.395963 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:24.396281 (+ 318us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:24.397063 (+ 782us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:08:24.397093 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:08:24.399329 (+ 2236us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:08:24.403077 (+ 3748us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:08:24.403087 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:08:24.403102 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:08:24.403585 (+ 483us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:08:24.404042 (+ 457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:08:24.404046 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:08:24.404049 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:08:24.404270 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:08:24.404933 (+ 663us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:08:24.404941 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:08:24.405311 (+ 370us) client_negotiation.cc:769] Sending connection context
0811 09:08:24.405578 (+ 267us) client_negotiation.cc:240] Negotiation successful
0811 09:08:24.406020 (+ 442us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":572,"thread_start_us":375,"threads_started":1}
I20260811 09:08:24.408795 8103 heartbeater.cc:347] Connected to a master server at 127.28.154.190:39219
I20260811 09:08:24.409108 8103 heartbeater.cc:464] Registering TS with master...
I20260811 09:08:24.409885 8103 heartbeater.cc:511] Master 127.28.154.190:39219 requested a full tablet report, sending...
I20260811 09:08:24.411415 7493 ts_manager.cc:194] Registered new tserver with Master: 96487104291043f2bf82fa52fe7a1e33 (127.28.154.130:46459)
I20260811 09:08:24.412281 7493 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.154.130@KRBTEST.COM'} at 127.28.154.130:59377
WARNING: no policy specified for kudu/127.28.154.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.154.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.154.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.131.keytab.
Entry for principal kudu/127.28.154.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.131.keytab.
I20260811 09:08:24.452152 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439250192320-29290-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439250192320-29290-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439250192320-29290-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439250192320-29290-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.154.131:0
--local_ip_for_outbound_sockets=127.28.154.131
--webserver_interface=127.28.154.131
--webserver_port=0
--tserver_master_addrs=127.28.154.190:39219
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.131.keytab
--principal=kudu/127.28.154.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:35053
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:08:24.781179 8111 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:08:24.781468 8111 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:08:24.781562 8111 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:08:24.788784 8111 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:08:24.788889 8111 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:08:24.789008 8111 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.154.131
I20260811 09:08:24.799760 8111 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:35053
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439250192320-29290-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439250192320-29290-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.131.keytab
--principal=kudu/127.28.154.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.154.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439250192320-29290-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.154.131
--webserver_port=0
--tserver_master_addrs=127.28.154.190:39219
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.154.131
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439250192320-29290-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:08:24.801393 8111 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:08:24.802767 8111 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:08:24.815299 8124 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:08:24.815382 8127 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:08:24.815510 8130 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:08:24.816212 8111 server_base.cc:1034] running on GCE node
I20260811 09:08:24.816886 8111 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:08:24.817936 8111 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:08:24.819200 8111 hybrid_clock.cc:648] HybridClock initialized: now 1786439304819150 us; error 40 us; skew 500 ppm
I20260811 09:08:24.819573 8111 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:08:24 dist-test-slave-xt4k krb5kdc[7395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439304, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:08:24.822890 8111 init.cc:377] Logged in from keytab as kudu/127.28.154.131@KRBTEST.COM (short username kudu)
I20260811 09:08:24.824425 8111 webserver.cc:460] Webserver started at http://127.28.154.131:38085/ using document root <none> and password file <none>
I20260811 09:08:24.825146 8111 fs_manager.cc:359] Metadata directory not provided
I20260811 09:08:24.825264 8111 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:08:24.825601 8111 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:08:24.828647 8111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439250192320-29290-0/minicluster-data/ts-2/data/instance:
uuid: "eae512880f424408872eb37d6f2f4fa9"
format_stamp: "Formatted at 2026-08-11 09:08:24 on dist-test-slave-xt4k"
server_key: "f1a26cdeb827032f6616f00d956003c0"
server_key_iv: "bd400041af134ef02cb057b5d4428015"
server_key_version: "encryptionkey@0"
I20260811 09:08:24.829437 8111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439250192320-29290-0/minicluster-data/ts-2/wal/instance:
uuid: "eae512880f424408872eb37d6f2f4fa9"
format_stamp: "Formatted at 2026-08-11 09:08:24 on dist-test-slave-xt4k"
server_key: "f1a26cdeb827032f6616f00d956003c0"
server_key_iv: "bd400041af134ef02cb057b5d4428015"
server_key_version: "encryptionkey@0"
I20260811 09:08:24.834770 8111 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 09:08:24.838253 8144 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:08:24.839706 8111 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 09:08:24.839869 8111 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439250192320-29290-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439250192320-29290-0/minicluster-data/ts-2/wal
uuid: "eae512880f424408872eb37d6f2f4fa9"
format_stamp: "Formatted at 2026-08-11 09:08:24 on dist-test-slave-xt4k"
server_key: "f1a26cdeb827032f6616f00d956003c0"
server_key_iv: "bd400041af134ef02cb057b5d4428015"
server_key_version: "encryptionkey@0"
I20260811 09:08:24.840068 8111 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439250192320-29290-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439250192320-29290-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439250192320-29290-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:08:24.858530 8111 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:08:24.862962 8111 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:08:24.863255 8111 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:08:24.864229 8111 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:08:24.865669 8111 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:08:24.865751 8111 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:08:24.865875 8111 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:08:24.865929 8111 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:08:24.876008 7838 heartbeater.cc:503] Master 127.28.154.190:39219 was elected leader, sending a full tablet report...
I20260811 09:08:24.896551 8111 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.131:45103
I20260811 09:08:24.896626 8356 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.131:45103 every 8 connection(s)
I20260811 09:08:24.897800 8111 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439250192320-29290-0/minicluster-data/ts-2/data/info.pb
I20260811 09:08:24.906177 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 8111
I20260811 09:08:24.906360 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439250192320-29290-0/minicluster-data/ts-2/wal/instance
I20260811 09:08:24.906821 29290 external_mini_cluster.cc:1428] Setting key db8846f4920d29054c3cda27bf4a29ea
Aug 11 09:08:24 dist-test-slave-xt4k krb5kdc[7395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439304, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.131@KRBTEST.COM for kudu/127.28.154.190@KRBTEST.COM
I20260811 09:08:24.917182 8106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:24.900295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.131:51521 (local address 127.28.154.190:39219)
0811 09:08:24.900515 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:24.900520 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:24.901579 (+ 1059us) server_negotiation.cc:407] Connection header received
0811 09:08:24.903454 (+ 1875us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:24.903460 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:24.903550 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:24.903637 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:24.905173 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:24.905925 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:24.907610 (+ 1685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:24.908140 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:24.908249 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:24.911822 (+ 3573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:24.911828 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:24.911831 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:08:24.911876 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:24.914326 (+ 2450us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:24.915031 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:24.915037 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:24.915040 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:24.915182 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:24.915576 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:24.915579 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:24.915581 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:24.915852 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:24.915985 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:24.916747 (+ 762us) server_negotiation.cc:299] Negotiation successful
0811 09:08:24.916958 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260811 09:08:24.918262 8361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:24.900704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:39219 (local address 127.28.154.131:51521)
0811 09:08:24.901420 (+ 716us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:24.901458 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:24.903172 (+ 1714us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:24.903881 (+ 709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:24.903889 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:24.904441 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:08:24.905008 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:24.905027 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:24.906079 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:24.906083 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:24.907457 (+ 1374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:24.907473 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:24.908413 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:24.908421 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:24.908646 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:24.909356 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:08:24.909384 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:08:24.911599 (+ 2215us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:08:24.914456 (+ 2857us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:08:24.914463 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:08:24.914479 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:08:24.914899 (+ 420us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:08:24.915284 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:08:24.915287 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:08:24.915296 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:08:24.915487 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:08:24.915964 (+ 477us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:08:24.915985 (+ 21us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:08:24.916382 (+ 397us) client_negotiation.cc:769] Sending connection context
0811 09:08:24.916640 (+ 258us) client_negotiation.cc:240] Negotiation successful
0811 09:08:24.917077 (+ 437us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":301,"threads_started":1}
I20260811 09:08:24.919834 8358 heartbeater.cc:347] Connected to a master server at 127.28.154.190:39219
I20260811 09:08:24.920154 8358 heartbeater.cc:464] Registering TS with master...
I20260811 09:08:24.921029 8358 heartbeater.cc:511] Master 127.28.154.190:39219 requested a full tablet report, sending...
I20260811 09:08:24.922598 7493 ts_manager.cc:194] Registered new tserver with Master: eae512880f424408872eb37d6f2f4fa9 (127.28.154.131:45103)
I20260811 09:08:24.923395 7493 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.154.131@KRBTEST.COM'} at 127.28.154.131:51521
I20260811 09:08:24.934629 29290 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439250192320-29290-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439250192320-29290-0/krb5kdc/test-admin.keytab.
Aug 11 09:08:24 dist-test-slave-xt4k krb5kdc[7395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439304, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:08:24.964921 29290 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260811 09:08:24.970079 8368 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:08:24.967528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43406 (local address 127.28.154.129:38515)
0811 09:08:24.968126 (+ 598us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:24.968134 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:24.968161 (+ 27us) server_negotiation.cc:407] Connection header received
0811 09:08:24.968223 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:24.968230 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:24.968507 (+ 277us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:24.968638 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:24.969644 (+ 1006us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43406: BlockingRecv error: recv got EOF from 127.0.0.1:43406 (error 108)
Metrics: {"server-negotiator.queue_time_us":451,"thread_start_us":261,"threads_started":1}
W20260811 09:08:24.970098 29290 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439250192320-29290-0/krb5kdc/krb5cc' of type FILE
Aug 11 09:08:24 dist-test-slave-xt4k krb5kdc[7395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439304, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:08:24.971643 29290 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 09:08:24 dist-test-slave-xt4k krb5kdc[7395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439304, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:08:24.973047 29290 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 09:08:24 dist-test-slave-xt4k krb5kdc[7395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439304, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.154.129@KRBTEST.COM
I20260811 09:08:24.988118 8368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:24.974568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43412 (local address 127.28.154.129:38515)
0811 09:08:24.974755 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:24.974761 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:24.974992 (+ 231us) server_negotiation.cc:407] Connection header received
0811 09:08:24.975112 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:24.975115 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:24.975174 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:24.975254 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:24.976132 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:24.977181 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:24.978330 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:24.978940 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:24.979044 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:24.982582 (+ 3538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:24.982613 (+ 31us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:24.982630 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:08:24.982671 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:24.985458 (+ 2787us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:24.986134 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:24.986140 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:24.986143 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:24.986258 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:24.986588 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:24.986591 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:24.986613 (+ 22us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:24.986976 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:24.987203 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:24.987533 (+ 330us) server_negotiation.cc:299] Negotiation successful
0811 09:08:24.987794 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
W20260811 09:08:24.994642 8368 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:08:24.992294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43416 (local address 127.28.154.129:38515)
0811 09:08:24.992513 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:24.992519 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:24.992664 (+ 145us) server_negotiation.cc:407] Connection header received
0811 09:08:24.992866 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:24.992871 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:24.992959 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:24.993035 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:24.994386 (+ 1351us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43416: BlockingRecv error: recv got EOF from 127.0.0.1:43416 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
W20260811 09:08:24.995175 29290 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439250192320-29290-0/krb5kdc/krb5cc' of type FILE
Aug 11 09:08:24 dist-test-slave-xt4k krb5kdc[7395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439304, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:08:24.997079 29290 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 09:08:24 dist-test-slave-xt4k krb5kdc[7395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439304, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:08:24.998723 29290 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 09:08:25 dist-test-slave-xt4k krb5kdc[7395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439304, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.154.129@KRBTEST.COM
I20260811 09:08:25.017004 8368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:25.000287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43418 (local address 127.28.154.129:38515)
0811 09:08:25.000486 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:25.000492 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:25.000817 (+ 325us) server_negotiation.cc:407] Connection header received
0811 09:08:25.000955 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:25.000960 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:25.001037 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:25.001131 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:25.002338 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:25.003148 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:25.004565 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:25.005042 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:25.005172 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:25.009412 (+ 4240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:25.009418 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:25.009421 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:08:25.009469 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:25.013502 (+ 4033us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:25.014590 (+ 1088us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:25.014597 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:25.014601 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:25.014787 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:25.015317 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:25.015323 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:25.015327 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:25.015732 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:25.015947 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:25.016406 (+ 459us) server_negotiation.cc:299] Negotiation successful
0811 09:08:25.016658 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
W20260811 09:08:25.022228 8368 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:08:25.020150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43428 (local address 127.28.154.129:38515)
0811 09:08:25.020359 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:25.020365 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:25.020672 (+ 307us) server_negotiation.cc:407] Connection header received
0811 09:08:25.020841 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:25.020847 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:25.020945 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:25.021036 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:25.022011 (+ 975us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43428: BlockingRecv error: recv got EOF from 127.0.0.1:43428 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
W20260811 09:08:25.023016 29290 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786439250192320-29290-0/krb5kdc/krb5cc' of type FILE
Aug 11 09:08:25 dist-test-slave-xt4k krb5kdc[7395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439305, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:08:25.025528 29290 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 09:08:25 dist-test-slave-xt4k krb5kdc[7395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439305, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:08:25.027328 29290 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 09:08:25 dist-test-slave-xt4k krb5kdc[7395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439305, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.154.129@KRBTEST.COM
I20260811 09:08:25.045292 8368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:25.028707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43430 (local address 127.28.154.129:38515)
0811 09:08:25.028864 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:25.028870 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:25.029307 (+ 437us) server_negotiation.cc:407] Connection header received
0811 09:08:25.029424 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:25.029427 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:25.029501 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:25.029580 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:25.030718 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:25.031873 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:25.033662 (+ 1789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:25.034326 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:25.034451 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:25.038942 (+ 4491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:25.038948 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:25.038953 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:08:25.038987 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:25.042136 (+ 3149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:25.043000 (+ 864us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:25.043007 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:25.043011 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:25.043221 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:25.043694 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:25.043700 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:25.043704 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:25.044080 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:25.044418 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:25.044737 (+ 319us) server_negotiation.cc:299] Negotiation successful
0811 09:08:25.044936 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260811 09:08:25.047039 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 7601
I20260811 09:08:25.058557 8368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:25.047779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43440 (local address 127.28.154.129:38515)
0811 09:08:25.047965 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:25.047970 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:25.048233 (+ 263us) server_negotiation.cc:407] Connection header received
0811 09:08:25.048403 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:25.048408 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:25.048478 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:25.048557 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:25.049445 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:25.050124 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:25.051285 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:25.051691 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:25.051806 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:25.053807 (+ 2001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:25.053812 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:25.053815 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:08:25.053844 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:25.056101 (+ 2257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:25.056763 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:25.056768 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:25.056772 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:25.056912 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:25.057257 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:25.057262 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:25.057265 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:25.057514 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:25.057752 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:25.058060 (+ 308us) server_negotiation.cc:299] Negotiation successful
0811 09:08:25.058255 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 09:08:25.059094 7827 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:08:25.143503 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 7601
I20260811 09:08:25.167021 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 7850
Aug 11 09:08:25 dist-test-slave-xt4k krb5kdc[7395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439305, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.154.130@KRBTEST.COM
I20260811 09:08:25.183456 8391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:25.167898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37620 (local address 127.28.154.130:46459)
0811 09:08:25.168471 (+ 573us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:25.168478 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:25.168505 (+ 27us) server_negotiation.cc:407] Connection header received
0811 09:08:25.168571 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:25.168577 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:25.168778 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:25.168928 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:25.169915 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:25.170967 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:25.172194 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:25.173029 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:25.173149 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:25.177338 (+ 4189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:25.177351 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:25.177368 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:08:25.177406 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:25.180436 (+ 3030us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:25.181213 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:25.181221 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:25.181224 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:25.181368 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:25.181775 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:25.181779 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:25.181781 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:25.182161 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:25.182508 (+ 347us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:25.182841 (+ 333us) server_negotiation.cc:299] Negotiation successful
0811 09:08:25.183137 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":411,"thread_start_us":279,"threads_started":1}
I20260811 09:08:25.184849 8093 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:08:25.268918 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 7850
I20260811 09:08:25.289134 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 8111
Aug 11 09:08:25 dist-test-slave-xt4k krb5kdc[7395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439305, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.154.131@KRBTEST.COM
I20260811 09:08:25.305044 8397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:25.290155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40702 (local address 127.28.154.131:45103)
0811 09:08:25.290794 (+ 639us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:25.290802 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:25.290834 (+ 32us) server_negotiation.cc:407] Connection header received
0811 09:08:25.290906 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:25.290913 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:25.291127 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:25.291257 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:25.292362 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:25.293466 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:25.294783 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:25.295398 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:25.295511 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:25.299381 (+ 3870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:25.299396 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:25.299414 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:08:25.299451 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:25.302255 (+ 2804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:25.302979 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:25.302987 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:25.302990 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:25.303128 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:25.303491 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:25.303495 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:25.303498 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:25.303888 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:25.304145 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:25.304445 (+ 300us) server_negotiation.cc:299] Negotiation successful
0811 09:08:25.304660 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":487,"thread_start_us":310,"threads_started":1}
I20260811 09:08:25.306303 8348 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:08:25.391214 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 8111
I20260811 09:08:25.413125 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 7422
I20260811 09:08:25.414482 7548 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:08:25.518911 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 7422
2026-08-11T09:08:25Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (4347 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:08:25 dist-test-slave-xt4k krb5kdc[8413](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:08:25 dist-test-slave-xt4k krb5kdc[8413](info): set up 2 sockets
Aug 11 09:08:25 dist-test-slave-xt4k krb5kdc[8413](info): commencing operation
krb5kdc: starting...
W20260811 09:08:26.813341 29290 mini_kdc.cc:121] Time spent starting KDC: real 1.244s user 0.000s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/krb5kdc/test-user.keytab.
Aug 11 09:08:26 dist-test-slave-xt4k krb5kdc[8413](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439306, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:08:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:08:26Z Disabled control of system clock
WARNING: no policy specified for oryx/127.28.154.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.28.154.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.28.154.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/krb5kdc/oryx_127.28.154.190.keytab.
Entry for principal oryx/127.28.154.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/krb5kdc/oryx_127.28.154.190.keytab.
I20260811 09:08:27.067718 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.154.190:42507
--webserver_interface=127.28.154.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/krb5kdc/oryx_127.28.154.190.keytab
--principal=oryx/127.28.154.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:33603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.154.190:42507
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:08:27.414855 8445 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:08:27.415186 8445 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:08:27.415333 8445 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:08:27.422760 8445 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:08:27.422895 8445 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:08:27.422956 8445 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:08:27.423038 8445 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:08:27.423101 8445 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:08:27.433450 8445 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:33603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.154.190:42507
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/krb5kdc/oryx_127.28.154.190.keytab
--principal=oryx/127.28.154.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.154.190:42507
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.154.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:08:27.434854 8445 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:08:27.436165 8445 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:08:27.448424 8455 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:08:27.448454 8450 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:08:27.448480 8451 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:08:27.449746 8445 server_base.cc:1034] running on GCE node
I20260811 09:08:27.450580 8445 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:08:27.452090 8445 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:08:27.453326 8445 hybrid_clock.cc:648] HybridClock initialized: now 1786439307453249 us; error 60 us; skew 500 ppm
I20260811 09:08:27.453723 8445 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:08:27 dist-test-slave-xt4k krb5kdc[8413](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439307, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.154.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:08:27.457052 8445 init.cc:377] Logged in from keytab as oryx/127.28.154.190@KRBTEST.COM (short username oryx)
I20260811 09:08:27.458690 8445 webserver.cc:460] Webserver started at http://127.28.154.190:38395/ using document root <none> and password file <none>
I20260811 09:08:27.459411 8445 fs_manager.cc:359] Metadata directory not provided
I20260811 09:08:27.459533 8445 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:08:27.459847 8445 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:08:27.463052 8445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/master-0/data/instance:
uuid: "41f9d60d712e4276b0da79549a2090e5"
format_stamp: "Formatted at 2026-08-11 09:08:27 on dist-test-slave-xt4k"
server_key: "5fb752a8642859c0fe0d70c5871c4c71"
server_key_iv: "fc76c86b1fd9c740e0381d4540acba4c"
server_key_version: "encryptionkey@0"
I20260811 09:08:27.463841 8445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/master-0/wal/instance:
uuid: "41f9d60d712e4276b0da79549a2090e5"
format_stamp: "Formatted at 2026-08-11 09:08:27 on dist-test-slave-xt4k"
server_key: "5fb752a8642859c0fe0d70c5871c4c71"
server_key_iv: "fc76c86b1fd9c740e0381d4540acba4c"
server_key_version: "encryptionkey@0"
I20260811 09:08:27.469453 8445 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260811 09:08:27.473869 8467 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:08:27.475522 8445 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260811 09:08:27.475713 8445 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/master-0/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/master-0/wal
uuid: "41f9d60d712e4276b0da79549a2090e5"
format_stamp: "Formatted at 2026-08-11 09:08:27 on dist-test-slave-xt4k"
server_key: "5fb752a8642859c0fe0d70c5871c4c71"
server_key_iv: "fc76c86b1fd9c740e0381d4540acba4c"
server_key_version: "encryptionkey@0"
I20260811 09:08:27.475927 8445 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:08:27.495568 8445 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:08:27.499881 8445 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:08:27.500183 8445 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:08:27.517720 8445 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.190:42507
I20260811 09:08:27.517742 8557 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.190:42507 every 8 connection(s)
I20260811 09:08:27.519119 8445 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
I20260811 09:08:27.522893 8558 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:08:27.526005 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 8445
I20260811 09:08:27.526217 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/master-0/wal/instance
I20260811 09:08:27.526871 29290 external_mini_cluster.cc:1428] Setting key 759d78824e0273ead4275aefad36665b
I20260811 09:08:27.536597 8558 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 41f9d60d712e4276b0da79549a2090e5: Bootstrap starting.
I20260811 09:08:27.541131 8558 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 41f9d60d712e4276b0da79549a2090e5: Neither blocks nor log segments found. Creating new log.
I20260811 09:08:27.542217 8558 log.cc:824] T 00000000000000000000000000000000 P 41f9d60d712e4276b0da79549a2090e5: Log is configured to *not* fsync() on all Append() calls
Aug 11 09:08:27 dist-test-slave-xt4k krb5kdc[8413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439306, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM
I20260811 09:08:27.545917 8558 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 41f9d60d712e4276b0da79549a2090e5: No bootstrap required, opened a new log
I20260811 09:08:27.553766 8563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:27.529007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44704 (local address 127.28.154.190:42507)
0811 09:08:27.529825 (+ 818us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:27.529842 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:27.529892 (+ 50us) server_negotiation.cc:407] Connection header received
0811 09:08:27.531971 (+ 2079us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:27.532007 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:27.532610 (+ 603us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:27.533101 (+ 491us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:27.534485 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:27.536525 (+ 2040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:27.537879 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:27.539083 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:27.539289 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:27.544704 (+ 5415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:27.544718 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:27.544739 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:08:27.544791 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:27.548780 (+ 3989us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:27.549584 (+ 804us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:27.549593 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:27.549604 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:27.549820 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:27.550266 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:27.550271 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:27.550275 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:27.551046 (+ 771us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:27.551379 (+ 333us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:27.551810 (+ 431us) server_negotiation.cc:299] Negotiation successful
0811 09:08:27.552283 (+ 473us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":575,"thread_start_us":333,"threads_started":1}
I20260811 09:08:27.556511 8558 raft_consensus.cc:348] T 00000000000000000000000000000000 P 41f9d60d712e4276b0da79549a2090e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41f9d60d712e4276b0da79549a2090e5" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 42507 } }
I20260811 09:08:27.556936 8558 raft_consensus.cc:374] T 00000000000000000000000000000000 P 41f9d60d712e4276b0da79549a2090e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:08:27.557045 8558 raft_consensus.cc:729] T 00000000000000000000000000000000 P 41f9d60d712e4276b0da79549a2090e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41f9d60d712e4276b0da79549a2090e5, State: Initialized, Role: FOLLOWER
I20260811 09:08:27.557646 8558 consensus_queue.cc:260] T 00000000000000000000000000000000 P 41f9d60d712e4276b0da79549a2090e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41f9d60d712e4276b0da79549a2090e5" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 42507 } }
I20260811 09:08:27.557971 8558 raft_consensus.cc:388] T 00000000000000000000000000000000 P 41f9d60d712e4276b0da79549a2090e5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:08:27.558144 8558 raft_consensus.cc:482] T 00000000000000000000000000000000 P 41f9d60d712e4276b0da79549a2090e5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:08:27.558308 8558 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 41f9d60d712e4276b0da79549a2090e5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:08:27.560667 8558 raft_consensus.cc:504] T 00000000000000000000000000000000 P 41f9d60d712e4276b0da79549a2090e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41f9d60d712e4276b0da79549a2090e5" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 42507 } }
I20260811 09:08:27.561201 8558 leader_election.cc:302] T 00000000000000000000000000000000 P 41f9d60d712e4276b0da79549a2090e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 41f9d60d712e4276b0da79549a2090e5; no voters:
I20260811 09:08:27.561918 8558 leader_election.cc:288] T 00000000000000000000000000000000 P 41f9d60d712e4276b0da79549a2090e5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:08:27.562088 8566 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 41f9d60d712e4276b0da79549a2090e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:08:27.563191 8566 raft_consensus.cc:686] T 00000000000000000000000000000000 P 41f9d60d712e4276b0da79549a2090e5 [term 1 LEADER]: Becoming Leader. State: Replica: 41f9d60d712e4276b0da79549a2090e5, State: Running, Role: LEADER
I20260811 09:08:27.564162 8566 consensus_queue.cc:237] T 00000000000000000000000000000000 P 41f9d60d712e4276b0da79549a2090e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41f9d60d712e4276b0da79549a2090e5" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 42507 } }
I20260811 09:08:27.564577 8558 sys_catalog.cc:564] T 00000000000000000000000000000000 P 41f9d60d712e4276b0da79549a2090e5 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:08:27.567135 8567 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41f9d60d712e4276b0da79549a2090e5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "41f9d60d712e4276b0da79549a2090e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41f9d60d712e4276b0da79549a2090e5" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 42507 } } }
I20260811 09:08:27.567567 8567 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41f9d60d712e4276b0da79549a2090e5 [sys.catalog]: This master's current role is: LEADER
I20260811 09:08:27.567723 8570 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41f9d60d712e4276b0da79549a2090e5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 41f9d60d712e4276b0da79549a2090e5. Latest consensus state: current_term: 1 leader_uuid: "41f9d60d712e4276b0da79549a2090e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41f9d60d712e4276b0da79549a2090e5" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 42507 } } }
I20260811 09:08:27.568063 8570 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41f9d60d712e4276b0da79549a2090e5 [sys.catalog]: This master's current role is: LEADER
I20260811 09:08:27.568852 8580 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:08:27.575255 8580 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:08:27.584478 8580 catalog_manager.cc:1347] Generated new cluster ID: 039df53890a0413897b986c5c7822102
I20260811 09:08:27.584609 8580 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:08:27.594736 8580 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:08:27.596746 8580 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:08:27.606503 8580 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 41f9d60d712e4276b0da79549a2090e5: Generated new TSK 0
I20260811 09:08:27.607559 8580 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.28.154.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.28.154.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.28.154.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/krb5kdc/oryx_127.28.154.129.keytab.
Entry for principal oryx/127.28.154.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/krb5kdc/oryx_127.28.154.129.keytab.
I20260811 09:08:27.672950 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.154.129:0
--local_ip_for_outbound_sockets=127.28.154.129
--webserver_interface=127.28.154.129
--webserver_port=0
--tserver_master_addrs=127.28.154.190:42507
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/krb5kdc/oryx_127.28.154.129.keytab
--principal=oryx/127.28.154.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:33603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:08:28.029122 8614 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:08:28.029435 8614 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:08:28.029577 8614 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:08:28.037220 8614 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:08:28.037386 8614 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:08:28.037575 8614 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.154.129
I20260811 09:08:28.048506 8614 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:33603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/krb5kdc/oryx_127.28.154.129.keytab
--principal=oryx/127.28.154.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.154.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.154.129
--webserver_port=0
--tserver_master_addrs=127.28.154.190:42507
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.154.129
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:08:28.049942 8614 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:08:28.051363 8614 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:08:28.065161 8622 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:08:28.065114 8624 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:08:28.065830 8621 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:08:28.065919 8614 server_base.cc:1034] running on GCE node
I20260811 09:08:28.066597 8614 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:08:28.067612 8614 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:08:28.068825 8614 hybrid_clock.cc:648] HybridClock initialized: now 1786439308068770 us; error 43 us; skew 500 ppm
I20260811 09:08:28.069159 8614 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:08:28 dist-test-slave-xt4k krb5kdc[8413](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439308, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.154.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:08:28.072760 8614 init.cc:377] Logged in from keytab as oryx/127.28.154.129@KRBTEST.COM (short username oryx)
I20260811 09:08:28.074285 8614 webserver.cc:460] Webserver started at http://127.28.154.129:37359/ using document root <none> and password file <none>
I20260811 09:08:28.075002 8614 fs_manager.cc:359] Metadata directory not provided
I20260811 09:08:28.075124 8614 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:08:28.075421 8614 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:08:28.078728 8614 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/ts-0/data/instance:
uuid: "2ed5f79097f84d8ea5faca70a459d1ca"
format_stamp: "Formatted at 2026-08-11 09:08:28 on dist-test-slave-xt4k"
server_key: "d14c96be2fd72fe57ec905c2353e10c3"
server_key_iv: "94def2a0bae551c8ba4a71d186c86e2d"
server_key_version: "encryptionkey@0"
I20260811 09:08:28.079680 8614 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/ts-0/wal/instance:
uuid: "2ed5f79097f84d8ea5faca70a459d1ca"
format_stamp: "Formatted at 2026-08-11 09:08:28 on dist-test-slave-xt4k"
server_key: "d14c96be2fd72fe57ec905c2353e10c3"
server_key_iv: "94def2a0bae551c8ba4a71d186c86e2d"
server_key_version: "encryptionkey@0"
I20260811 09:08:28.086162 8614 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.003s
I20260811 09:08:28.090298 8634 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:08:28.092681 8614 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 09:08:28.092994 8614 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/ts-0/wal
uuid: "2ed5f79097f84d8ea5faca70a459d1ca"
format_stamp: "Formatted at 2026-08-11 09:08:28 on dist-test-slave-xt4k"
server_key: "d14c96be2fd72fe57ec905c2353e10c3"
server_key_iv: "94def2a0bae551c8ba4a71d186c86e2d"
server_key_version: "encryptionkey@0"
I20260811 09:08:28.093300 8614 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:08:28.119428 8614 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:08:28.123875 8614 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:08:28.124140 8614 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:08:28.125118 8614 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:08:28.126452 8614 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:08:28.126530 8614 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:08:28.126626 8614 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:08:28.126672 8614 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:08:28.159914 8614 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.129:34797
I20260811 09:08:28.159889 8849 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.129:34797 every 8 connection(s)
I20260811 09:08:28.161819 8614 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/ts-0/data/info.pb
I20260811 09:08:28.163439 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 8614
I20260811 09:08:28.163656 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/ts-0/wal/instance
I20260811 09:08:28.164587 29290 external_mini_cluster.cc:1428] Setting key fb66bc9405fd05cf54e32fe81f143ae9
Aug 11 09:08:28 dist-test-slave-xt4k krb5kdc[8413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439308, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.154.129@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM
I20260811 09:08:28.185251 8854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:28.164677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.129:51359 (local address 127.28.154.190:42507)
0811 09:08:28.165607 (+ 930us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:28.165613 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:28.166485 (+ 872us) server_negotiation.cc:407] Connection header received
0811 09:08:28.168996 (+ 2511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:28.169001 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:28.169154 (+ 153us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:28.169247 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:28.171482 (+ 2235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:28.172247 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:28.174432 (+ 2185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:28.174979 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:28.175099 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:28.179254 (+ 4155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:28.179261 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:28.179266 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:08:28.179308 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:28.182136 (+ 2828us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:28.182897 (+ 761us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:28.182902 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:28.182904 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:28.183025 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:28.183446 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:28.183449 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:28.183452 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:28.183732 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:28.183865 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:28.184650 (+ 785us) server_negotiation.cc:299] Negotiation successful
0811 09:08:28.184882 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":764,"thread_start_us":656,"threads_started":1}
I20260811 09:08:28.186275 8856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:28.165162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:42507 (local address 127.28.154.129:51359)
0811 09:08:28.166298 (+ 1136us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:28.166352 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:28.168660 (+ 2308us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:28.169530 (+ 870us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:28.169542 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:28.170459 (+ 917us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:08:28.171276 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:28.171302 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:28.172430 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:28.172435 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:28.174251 (+ 1816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:28.174267 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:28.175228 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:28.175235 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:28.175516 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:28.176250 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:08:28.176280 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:08:28.178980 (+ 2700us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:08:28.182291 (+ 3311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:08:28.182302 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:08:28.182322 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:08:28.182769 (+ 447us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:08:28.183130 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:08:28.183135 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:08:28.183139 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:08:28.183330 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:08:28.183858 (+ 528us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:08:28.183866 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:08:28.184227 (+ 361us) client_negotiation.cc:769] Sending connection context
0811 09:08:28.184516 (+ 289us) client_negotiation.cc:240] Negotiation successful
0811 09:08:28.185095 (+ 579us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":838,"thread_start_us":595,"threads_started":1}
I20260811 09:08:28.187885 8850 heartbeater.cc:347] Connected to a master server at 127.28.154.190:42507
I20260811 09:08:28.188261 8850 heartbeater.cc:464] Registering TS with master...
I20260811 09:08:28.189226 8850 heartbeater.cc:511] Master 127.28.154.190:42507 requested a full tablet report, sending...
WARNING: no policy specified for oryx/127.28.154.130@KRBTEST.COM; defaulting to no policy
I20260811 09:08:28.192481 8495 ts_manager.cc:194] Registered new tserver with Master: 2ed5f79097f84d8ea5faca70a459d1ca (127.28.154.129:34797)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.28.154.130@KRBTEST.COM" created.
I20260811 09:08:28.194085 8495 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.28.154.129@KRBTEST.COM'} at 127.28.154.129:51359
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.28.154.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/krb5kdc/oryx_127.28.154.130.keytab.
Entry for principal oryx/127.28.154.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/krb5kdc/oryx_127.28.154.130.keytab.
I20260811 09:08:28.224444 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.154.130:0
--local_ip_for_outbound_sockets=127.28.154.130
--webserver_interface=127.28.154.130
--webserver_port=0
--tserver_master_addrs=127.28.154.190:42507
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/krb5kdc/oryx_127.28.154.130.keytab
--principal=oryx/127.28.154.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:33603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:08:28.560947 8861 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:08:28.561218 8861 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:08:28.561340 8861 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:08:28.568750 8861 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:08:28.568861 8861 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:08:28.569005 8861 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.154.130
I20260811 09:08:28.578735 8861 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:33603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/krb5kdc/oryx_127.28.154.130.keytab
--principal=oryx/127.28.154.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.154.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.154.130
--webserver_port=0
--tserver_master_addrs=127.28.154.190:42507
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.154.130
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:08:28.580081 8861 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:08:28.581516 8861 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:08:28.594511 8867 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:08:28.594586 8868 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:08:28.594609 8871 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:08:28.595254 8861 server_base.cc:1034] running on GCE node
I20260811 09:08:28.595850 8861 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:08:28.596925 8861 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:08:28.598119 8861 hybrid_clock.cc:648] HybridClock initialized: now 1786439308598064 us; error 46 us; skew 500 ppm
I20260811 09:08:28.598448 8861 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:08:28 dist-test-slave-xt4k krb5kdc[8413](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439308, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.154.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:08:28.601667 8861 init.cc:377] Logged in from keytab as oryx/127.28.154.130@KRBTEST.COM (short username oryx)
I20260811 09:08:28.603183 8861 webserver.cc:460] Webserver started at http://127.28.154.130:35155/ using document root <none> and password file <none>
I20260811 09:08:28.603879 8861 fs_manager.cc:359] Metadata directory not provided
I20260811 09:08:28.603998 8861 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:08:28.604300 8861 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:08:28.607327 8861 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/ts-1/data/instance:
uuid: "5675cf4144864bd9aa773a78d2e2ed50"
format_stamp: "Formatted at 2026-08-11 09:08:28 on dist-test-slave-xt4k"
server_key: "a513d67546da7d095f1fae24fda2e4ec"
server_key_iv: "5153fafe851d0e0544e789ffd70c2b8e"
server_key_version: "encryptionkey@0"
I20260811 09:08:28.608114 8861 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/ts-1/wal/instance:
uuid: "5675cf4144864bd9aa773a78d2e2ed50"
format_stamp: "Formatted at 2026-08-11 09:08:28 on dist-test-slave-xt4k"
server_key: "a513d67546da7d095f1fae24fda2e4ec"
server_key_iv: "5153fafe851d0e0544e789ffd70c2b8e"
server_key_version: "encryptionkey@0"
I20260811 09:08:28.613507 8861 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260811 09:08:28.617386 8885 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:08:28.618916 8861 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 09:08:28.619091 8861 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/ts-1/wal
uuid: "5675cf4144864bd9aa773a78d2e2ed50"
format_stamp: "Formatted at 2026-08-11 09:08:28 on dist-test-slave-xt4k"
server_key: "a513d67546da7d095f1fae24fda2e4ec"
server_key_iv: "5153fafe851d0e0544e789ffd70c2b8e"
server_key_version: "encryptionkey@0"
I20260811 09:08:28.619292 8861 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:08:28.651250 8861 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:08:28.656176 8861 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:08:28.656538 8861 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:08:28.657559 8861 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:08:28.658972 8861 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:08:28.659052 8861 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:08:28.659173 8861 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:08:28.659232 8861 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:08:28.690323 8861 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.130:35019
I20260811 09:08:28.690363 9101 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.130:35019 every 8 connection(s)
I20260811 09:08:28.691632 8861 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/ts-1/data/info.pb
I20260811 09:08:28.699000 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 8861
I20260811 09:08:28.699232 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/ts-1/wal/instance
I20260811 09:08:28.699864 29290 external_mini_cluster.cc:1428] Setting key 8f39fc5f6cf057237535840ed788cec6
Aug 11 09:08:28 dist-test-slave-xt4k krb5kdc[8413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439308, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.154.130@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM
I20260811 09:08:28.714651 9104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:28.694351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.130:46075 (local address 127.28.154.190:42507)
0811 09:08:28.694789 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:28.694794 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:28.695817 (+ 1023us) server_negotiation.cc:407] Connection header received
0811 09:08:28.697698 (+ 1881us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:28.697704 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:28.697870 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:28.697988 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:28.700150 (+ 2162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:28.701055 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:28.703312 (+ 2257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:28.703799 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:28.703913 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:28.708181 (+ 4268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:28.708188 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:28.708191 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:08:28.708228 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:28.711026 (+ 2798us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:28.711943 (+ 917us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:28.711947 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:28.711949 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:28.712085 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:28.712559 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:28.712563 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:28.712565 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:28.713078 (+ 513us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:28.713245 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:28.714018 (+ 773us) server_negotiation.cc:299] Negotiation successful
0811 09:08:28.714261 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":222,"threads_started":1}
I20260811 09:08:28.715663 9105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:28.694903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:42507 (local address 127.28.154.130:46075)
0811 09:08:28.695646 (+ 743us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:28.695693 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:28.697420 (+ 1727us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:28.698296 (+ 876us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:28.698304 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:28.699035 (+ 731us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:08:28.699930 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:28.699960 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:28.701217 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:28.701222 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:28.703069 (+ 1847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:28.703087 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:28.704046 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:28.704054 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:28.704402 (+ 348us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:28.705131 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:08:28.705160 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:08:28.707893 (+ 2733us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:08:28.711220 (+ 3327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:08:28.711230 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:08:28.711250 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:08:28.711813 (+ 563us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:08:28.712200 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:08:28.712204 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:08:28.712206 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:08:28.712448 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:08:28.713236 (+ 788us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:08:28.713244 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:08:28.713623 (+ 379us) client_negotiation.cc:769] Sending connection context
0811 09:08:28.713878 (+ 255us) client_negotiation.cc:240] Negotiation successful
0811 09:08:28.714455 (+ 577us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":284,"threads_started":1}
I20260811 09:08:28.717247 9102 heartbeater.cc:347] Connected to a master server at 127.28.154.190:42507
I20260811 09:08:28.717554 9102 heartbeater.cc:464] Registering TS with master...
I20260811 09:08:28.718369 9102 heartbeater.cc:511] Master 127.28.154.190:42507 requested a full tablet report, sending...
I20260811 09:08:28.720181 8495 ts_manager.cc:194] Registered new tserver with Master: 5675cf4144864bd9aa773a78d2e2ed50 (127.28.154.130:35019)
I20260811 09:08:28.721371 8495 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.28.154.130@KRBTEST.COM'} at 127.28.154.130:46075
WARNING: no policy specified for oryx/127.28.154.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.28.154.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.28.154.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/krb5kdc/oryx_127.28.154.131.keytab.
Entry for principal oryx/127.28.154.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/krb5kdc/oryx_127.28.154.131.keytab.
I20260811 09:08:28.755548 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.154.131:0
--local_ip_for_outbound_sockets=127.28.154.131
--webserver_interface=127.28.154.131
--webserver_port=0
--tserver_master_addrs=127.28.154.190:42507
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/krb5kdc/oryx_127.28.154.131.keytab
--principal=oryx/127.28.154.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:33603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:08:29.095327 9116 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:08:29.095654 9116 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:08:29.095780 9116 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:08:29.103547 9116 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:08:29.103688 9116 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:08:29.103842 9116 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.154.131
I20260811 09:08:29.114640 9116 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:33603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/krb5kdc/oryx_127.28.154.131.keytab
--principal=oryx/127.28.154.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.154.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.154.131
--webserver_port=0
--tserver_master_addrs=127.28.154.190:42507
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.154.131
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:08:29.116046 9116 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:08:29.117852 9116 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:08:29.132288 9126 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:08:29.132400 9130 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:08:29.132366 9125 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:08:29.133325 9116 server_base.cc:1034] running on GCE node
I20260811 09:08:29.133968 9116 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:08:29.134935 9116 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:08:29.136193 9116 hybrid_clock.cc:648] HybridClock initialized: now 1786439309136137 us; error 42 us; skew 500 ppm
I20260811 09:08:29.136615 9116 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:08:29 dist-test-slave-xt4k krb5kdc[8413](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439309, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.154.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:08:29.140054 9116 init.cc:377] Logged in from keytab as oryx/127.28.154.131@KRBTEST.COM (short username oryx)
I20260811 09:08:29.142074 9116 webserver.cc:460] Webserver started at http://127.28.154.131:44757/ using document root <none> and password file <none>
I20260811 09:08:29.143039 9116 fs_manager.cc:359] Metadata directory not provided
I20260811 09:08:29.143188 9116 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:08:29.143536 9116 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:08:29.146714 9116 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/ts-2/data/instance:
uuid: "c62d163d6d3a435b821da93a68c125f9"
format_stamp: "Formatted at 2026-08-11 09:08:29 on dist-test-slave-xt4k"
server_key: "bfa6a7b4222541dc7f9e54f6487a457a"
server_key_iv: "5b7d63cc7db4beabbbd82b558b4eda63"
server_key_version: "encryptionkey@0"
I20260811 09:08:29.147550 9116 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/ts-2/wal/instance:
uuid: "c62d163d6d3a435b821da93a68c125f9"
format_stamp: "Formatted at 2026-08-11 09:08:29 on dist-test-slave-xt4k"
server_key: "bfa6a7b4222541dc7f9e54f6487a457a"
server_key_iv: "5b7d63cc7db4beabbbd82b558b4eda63"
server_key_version: "encryptionkey@0"
I20260811 09:08:29.153170 9116 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.003s
I20260811 09:08:29.157114 9139 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:08:29.158638 9116 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 09:08:29.158831 9116 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/ts-2/wal
uuid: "c62d163d6d3a435b821da93a68c125f9"
format_stamp: "Formatted at 2026-08-11 09:08:29 on dist-test-slave-xt4k"
server_key: "bfa6a7b4222541dc7f9e54f6487a457a"
server_key_iv: "5b7d63cc7db4beabbbd82b558b4eda63"
server_key_version: "encryptionkey@0"
I20260811 09:08:29.159042 9116 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:08:29.190323 9116 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:08:29.194804 9116 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:08:29.195101 9116 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:08:29.196120 9116 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:08:29.197566 9116 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:08:29.197557 8850 heartbeater.cc:503] Master 127.28.154.190:42507 was elected leader, sending a full tablet report...
I20260811 09:08:29.197666 9116 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:08:29.197789 9116 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:08:29.197862 9116 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:08:29.228317 9116 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.131:39329
I20260811 09:08:29.228425 9363 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.131:39329 every 8 connection(s)
I20260811 09:08:29.229657 9116 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/ts-2/data/info.pb
I20260811 09:08:29.231468 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 9116
I20260811 09:08:29.231688 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/ts-2/wal/instance
I20260811 09:08:29.232230 29290 external_mini_cluster.cc:1428] Setting key 958c8d9e080f6bf655b47edc62506f50
Aug 11 09:08:29 dist-test-slave-xt4k krb5kdc[8413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439309, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.154.131@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM
I20260811 09:08:29.251718 9373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:29.232824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.131:50003 (local address 127.28.154.190:42507)
0811 09:08:29.233547 (+ 723us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:29.233553 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:29.235157 (+ 1604us) server_negotiation.cc:407] Connection header received
0811 09:08:29.236551 (+ 1394us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:29.236556 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:29.236639 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:29.236733 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:29.238829 (+ 2096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:29.239666 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:29.241440 (+ 1774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:29.241850 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:29.241976 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:29.245543 (+ 3567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:29.245549 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:29.245553 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:08:29.245589 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:29.248574 (+ 2985us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:29.249373 (+ 799us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:29.249377 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:29.249380 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:29.249505 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:29.249982 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:29.249986 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:29.249988 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:29.250268 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:29.250389 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:29.251238 (+ 849us) server_negotiation.cc:299] Negotiation successful
0811 09:08:29.251419 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":602,"thread_start_us":484,"threads_started":1}
I20260811 09:08:29.252758 9377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:29.233472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:42507 (local address 127.28.154.131:50003)
0811 09:08:29.234322 (+ 850us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:29.234376 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:29.236237 (+ 1861us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:29.237346 (+ 1109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:29.237358 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:29.237959 (+ 601us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:08:29.238582 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:29.238603 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:29.239837 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:29.239841 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:29.241277 (+ 1436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:29.241293 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:29.242094 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:29.242100 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:29.242317 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:29.243032 (+ 715us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:08:29.243056 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:08:29.245324 (+ 2268us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:08:29.248737 (+ 3413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:08:29.248746 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:08:29.248761 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:08:29.249235 (+ 474us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:08:29.249632 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:08:29.249636 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:08:29.249639 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:08:29.249859 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:08:29.250422 (+ 563us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:08:29.250429 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:08:29.250847 (+ 418us) client_negotiation.cc:769] Sending connection context
0811 09:08:29.251089 (+ 242us) client_negotiation.cc:240] Negotiation successful
0811 09:08:29.251530 (+ 441us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":564,"thread_start_us":317,"threads_started":1}
I20260811 09:08:29.254516 9367 heartbeater.cc:347] Connected to a master server at 127.28.154.190:42507
I20260811 09:08:29.254858 9367 heartbeater.cc:464] Registering TS with master...
I20260811 09:08:29.255666 9367 heartbeater.cc:511] Master 127.28.154.190:42507 requested a full tablet report, sending...
I20260811 09:08:29.257184 8495 ts_manager.cc:194] Registered new tserver with Master: c62d163d6d3a435b821da93a68c125f9 (127.28.154.131:39329)
I20260811 09:08:29.258029 8495 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.28.154.131@KRBTEST.COM'} at 127.28.154.131:50003
I20260811 09:08:29.260720 29290 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.28.154.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.28.154.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.28.154.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/krb5kdc/oryx_127.28.154.189.keytab.
Entry for principal oryx/127.28.154.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/krb5kdc/oryx_127.28.154.189.keytab.
I20260811 09:08:29.319767 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.154.189:33355
--webserver_interface=127.28.154.189
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/krb5kdc/oryx_127.28.154.189.keytab
--principal=oryx/127.28.154.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:33603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.154.190:42507,127.28.154.189:33355
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:08:29.660076 9381 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:08:29.660445 9381 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:08:29.660583 9381 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:08:29.668936 9381 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:08:29.669070 9381 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:08:29.669162 9381 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:08:29.669238 9381 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:08:29.669297 9381 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:08:29.679967 9381 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:33603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.28.154.190:42507,127.28.154.189:33355
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/krb5kdc/oryx_127.28.154.189.keytab
--principal=oryx/127.28.154.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.154.189:33355
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.28.154.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
W20260811 09:08:29.680440 9381 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.28.154.190:42507,127.28.154.189:33355'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260811 09:08:29.681587 9381 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:08:29.683233 9381 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:08:29.695165 9387 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:08:29.695469 9390 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:08:29.695314 9395 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:08:29.697381 9381 server_base.cc:1034] running on GCE node
I20260811 09:08:29.698167 9381 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:08:29.699643 9381 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:08:29.700852 9381 hybrid_clock.cc:648] HybridClock initialized: now 1786439309700779 us; error 48 us; skew 500 ppm
I20260811 09:08:29.701174 9381 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:08:29 dist-test-slave-xt4k krb5kdc[8413](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439309, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.154.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:08:29.707194 9381 init.cc:377] Logged in from keytab as oryx/127.28.154.189@KRBTEST.COM (short username oryx)
I20260811 09:08:29.708765 9381 webserver.cc:460] Webserver started at http://127.28.154.189:45223/ using document root <none> and password file <none>
I20260811 09:08:29.709484 9381 fs_manager.cc:359] Metadata directory not provided
I20260811 09:08:29.709573 9381 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:08:29.709846 9381 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:08:29.712999 9381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/master-1/data/instance:
uuid: "169951c4d2524ade9f73f60b2d109e3a"
format_stamp: "Formatted at 2026-08-11 09:08:29 on dist-test-slave-xt4k"
server_key: "3f8296f511286c54d2d0a33238a8482c"
server_key_iv: "f740f8d7ab94a3489eed101465d39b47"
server_key_version: "encryptionkey@0"
I20260811 09:08:29.713783 9381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/master-1/wal/instance:
uuid: "169951c4d2524ade9f73f60b2d109e3a"
format_stamp: "Formatted at 2026-08-11 09:08:29 on dist-test-slave-xt4k"
server_key: "3f8296f511286c54d2d0a33238a8482c"
server_key_iv: "f740f8d7ab94a3489eed101465d39b47"
server_key_version: "encryptionkey@0"
I20260811 09:08:29.719700 9381 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260811 09:08:29.723536 9405 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:08:29.724251 9102 heartbeater.cc:503] Master 127.28.154.190:42507 was elected leader, sending a full tablet report...
I20260811 09:08:29.725198 9381 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 09:08:29.725365 9381 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/master-1/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/master-1/wal
uuid: "169951c4d2524ade9f73f60b2d109e3a"
format_stamp: "Formatted at 2026-08-11 09:08:29 on dist-test-slave-xt4k"
server_key: "3f8296f511286c54d2d0a33238a8482c"
server_key_iv: "f740f8d7ab94a3489eed101465d39b47"
server_key_version: "encryptionkey@0"
I20260811 09:08:29.725574 9381 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:08:29.750936 9381 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:08:29.755761 9381 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:08:29.756034 9381 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:08:29.774538 9381 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.189:33355
I20260811 09:08:29.774557 9502 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.189:33355 every 8 connection(s)
I20260811 09:08:29.775884 9381 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/master-1/data/info.pb
I20260811 09:08:29.779554 9504 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:08:29.784725 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 9381
I20260811 09:08:29.787096 9504 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.154.190" port: 42507 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 09:08:29 dist-test-slave-xt4k krb5kdc[8413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439309, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.154.189@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM
Aug 11 09:08:29 dist-test-slave-xt4k krb5kdc[8413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439306, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.28.154.189@KRBTEST.COM
I20260811 09:08:29.808429 9513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:29.789876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44708 (local address 127.28.154.190:42507)
0811 09:08:29.791188 (+ 1312us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:29.791194 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:29.791213 (+ 19us) server_negotiation.cc:407] Connection header received
0811 09:08:29.791276 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:29.791281 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:29.791376 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:29.791487 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:29.793223 (+ 1736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:29.794419 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:29.796573 (+ 2154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:29.797218 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:29.797373 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:29.801620 (+ 4247us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:29.801629 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:29.801634 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:08:29.801687 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:29.804674 (+ 2987us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:29.805656 (+ 982us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:29.805661 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:29.805663 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:29.805789 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:29.806251 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:29.806257 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:29.806260 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:29.806722 (+ 462us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:29.806835 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:29.807777 (+ 942us) server_negotiation.cc:299] Negotiation successful
0811 09:08:29.808017 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1196,"thread_start_us":1095,"threads_started":1}
I20260811 09:08:29.809880 9509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:29.786967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36032 (local address 127.28.154.189:33355)
0811 09:08:29.787907 (+ 940us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:29.787919 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:29.787964 (+ 45us) server_negotiation.cc:407] Connection header received
0811 09:08:29.790161 (+ 2197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:29.790192 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:29.790686 (+ 494us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:29.791300 (+ 614us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:29.793174 (+ 1874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:29.794802 (+ 1628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:29.796578 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:29.797423 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:29.797620 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:29.802674 (+ 5054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:29.802688 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:29.802710 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:08:29.802758 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:29.805865 (+ 3107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:29.806740 (+ 875us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:29.806748 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:29.806750 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:29.806913 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:29.807291 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:29.807296 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:29.807299 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:29.807760 (+ 461us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:29.808062 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:29.808412 (+ 350us) server_negotiation.cc:299] Negotiation successful
0811 09:08:29.808672 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":710,"thread_start_us":410,"threads_started":1}
I20260811 09:08:29.809911 9512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:29.789827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:42507 (local address 127.0.0.1:44708)
0811 09:08:29.790638 (+ 811us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:29.790687 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:29.790868 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:29.791607 (+ 739us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:29.791615 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:29.792480 (+ 865us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:08:29.793032 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:29.793059 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:29.794600 (+ 1541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:29.794604 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:29.796428 (+ 1824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:29.796442 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:29.797531 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:29.797541 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:29.797768 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:29.798526 (+ 758us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:08:29.798552 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:08:29.801377 (+ 2825us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:08:29.804818 (+ 3441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:08:29.804830 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:08:29.804845 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:08:29.805504 (+ 659us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:08:29.805892 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:08:29.805896 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:08:29.805900 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:08:29.806129 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:08:29.806849 (+ 720us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:08:29.806857 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:08:29.807297 (+ 440us) client_negotiation.cc:769] Sending connection context
0811 09:08:29.807675 (+ 378us) client_negotiation.cc:240] Negotiation successful
0811 09:08:29.808207 (+ 532us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":322,"threads_started":1}
I20260811 09:08:29.812909 9504 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.154.189" port: 33355 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 09:08:29 dist-test-slave-xt4k krb5kdc[8413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439309, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.154.189@KRBTEST.COM for oryx/127.28.154.189@KRBTEST.COM
I20260811 09:08:29.825196 9512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:29.813761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.189:33355 (local address 127.0.0.1:36048)
0811 09:08:29.813972 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:29.813991 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:29.814093 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:29.814435 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:29.814439 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:29.814811 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:08:29.814949 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:29.814957 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:29.815957 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:29.815961 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:29.816883 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:29.816895 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:29.817651 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:29.817655 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:29.817836 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:29.818340 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:08:29.818357 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:08:29.820322 (+ 1965us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:08:29.823036 (+ 2714us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:08:29.823040 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:08:29.823042 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:08:29.823490 (+ 448us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:08:29.823855 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:08:29.823859 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:08:29.823861 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:08:29.823994 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:08:29.824506 (+ 512us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:08:29.824509 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:08:29.824633 (+ 124us) client_negotiation.cc:769] Sending connection context
0811 09:08:29.824742 (+ 109us) client_negotiation.cc:240] Negotiation successful
0811 09:08:29.824953 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66}
I20260811 09:08:29.825475 9509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:29.814012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36048 (local address 127.28.154.189:33355)
0811 09:08:29.814180 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:29.814185 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:29.814199 (+ 14us) server_negotiation.cc:407] Connection header received
0811 09:08:29.814237 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:29.814240 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:29.814292 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:29.814367 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:29.815102 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:29.815824 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:29.817014 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:29.817415 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:29.817510 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:29.820511 (+ 3001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:29.820516 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:29.820519 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:08:29.820547 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:29.822895 (+ 2348us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:29.823611 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:29.823615 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:29.823617 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:29.823761 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:29.824091 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:29.824094 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:29.824096 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:29.824390 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:29.824499 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:29.824838 (+ 339us) server_negotiation.cc:299] Negotiation successful
0811 09:08:29.825106 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":80,"server-negotiator.queue_time_us":61}
I20260811 09:08:29.830222 9504 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 169951c4d2524ade9f73f60b2d109e3a: Bootstrap starting.
I20260811 09:08:29.833560 9504 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 169951c4d2524ade9f73f60b2d109e3a: Neither blocks nor log segments found. Creating new log.
I20260811 09:08:29.834537 9504 log.cc:824] T 00000000000000000000000000000000 P 169951c4d2524ade9f73f60b2d109e3a: Log is configured to *not* fsync() on all Append() calls
I20260811 09:08:29.837172 9504 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 169951c4d2524ade9f73f60b2d109e3a: No bootstrap required, opened a new log
I20260811 09:08:29.844305 9504 raft_consensus.cc:348] T 00000000000000000000000000000000 P 169951c4d2524ade9f73f60b2d109e3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41f9d60d712e4276b0da79549a2090e5" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 42507 } } peers { permanent_uuid: "169951c4d2524ade9f73f60b2d109e3a" member_type: VOTER last_known_addr { host: "127.28.154.189" port: 33355 } }
I20260811 09:08:29.844694 9504 raft_consensus.cc:374] T 00000000000000000000000000000000 P 169951c4d2524ade9f73f60b2d109e3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:08:29.844770 9504 raft_consensus.cc:729] T 00000000000000000000000000000000 P 169951c4d2524ade9f73f60b2d109e3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 169951c4d2524ade9f73f60b2d109e3a, State: Initialized, Role: FOLLOWER
I20260811 09:08:29.845268 9504 consensus_queue.cc:260] T 00000000000000000000000000000000 P 169951c4d2524ade9f73f60b2d109e3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41f9d60d712e4276b0da79549a2090e5" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 42507 } } peers { permanent_uuid: "169951c4d2524ade9f73f60b2d109e3a" member_type: VOTER last_known_addr { host: "127.28.154.189" port: 33355 } }
I20260811 09:08:29.846062 9515 sys_catalog.cc:455] T 00000000000000000000000000000000 P 169951c4d2524ade9f73f60b2d109e3a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41f9d60d712e4276b0da79549a2090e5" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 42507 } } peers { permanent_uuid: "169951c4d2524ade9f73f60b2d109e3a" member_type: VOTER last_known_addr { host: "127.28.154.189" port: 33355 } } }
I20260811 09:08:29.846367 9515 sys_catalog.cc:458] T 00000000000000000000000000000000 P 169951c4d2524ade9f73f60b2d109e3a [sys.catalog]: This master's current role is: FOLLOWER
I20260811 09:08:29.847041 9504 sys_catalog.cc:564] T 00000000000000000000000000000000 P 169951c4d2524ade9f73f60b2d109e3a [sys.catalog]: configured and running, proceeding with master startup.
W20260811 09:08:29.856204 9537 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 169951c4d2524ade9f73f60b2d109e3a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 09:08:29.856400 9537 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 09:08:29.860026 9381 master_runner.cc:428] Detected that this master 169951c4d2524ade9f73f60b2d109e3a is joining an existing cluster
I20260811 09:08:29.860141 9381 master_runner.cc:432] Initiating AddMaster RPC to add 127.28.154.189:33355
I20260811 09:08:29.891436 9509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:29.865537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36050 (local address 127.28.154.189:33355)
0811 09:08:29.865716 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:29.865722 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:29.876198 (+ 10476us) server_negotiation.cc:407] Connection header received
0811 09:08:29.876454 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:29.876459 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:29.876532 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:29.876678 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:29.878620 (+ 1942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:29.879350 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:29.881313 (+ 1963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:29.881963 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:29.882081 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:29.884435 (+ 2354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:29.884441 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:29.884445 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:08:29.884490 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:29.888257 (+ 3767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:29.889127 (+ 870us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:29.889132 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:29.889135 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:29.889302 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:29.889712 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:29.889717 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:29.889720 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:29.890173 (+ 453us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:29.890286 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:29.890790 (+ 504us) server_negotiation.cc:299] Negotiation successful
0811 09:08:29.891039 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":16,"server-negotiator.queue_time_us":40}
I20260811 09:08:29.892920 9546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:29.864930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.189:33355 (local address 127.0.0.1:36050)
0811 09:08:29.876075 (+ 11145us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:29.876095 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:29.876280 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:29.876893 (+ 613us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:29.876898 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:29.877816 (+ 918us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:08:29.878464 (+ 648us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:29.878477 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:29.879568 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:29.879571 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:29.881131 (+ 1560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:29.881147 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:29.882145 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:29.882152 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:29.882320 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:29.882968 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:08:29.882990 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:08:29.884108 (+ 1118us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:08:29.888440 (+ 4332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:08:29.888445 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:08:29.888449 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:08:29.888977 (+ 528us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:08:29.889434 (+ 457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:08:29.889438 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:08:29.889441 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:08:29.889599 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:08:29.890364 (+ 765us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:08:29.890368 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:08:29.890543 (+ 175us) client_negotiation.cc:769] Sending connection context
0811 09:08:29.892106 (+ 1563us) client_negotiation.cc:240] Negotiation successful
0811 09:08:29.892452 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10975,"mutex_wait_us":84,"spinlock_wait_cycles":65280,"thread_start_us":11175,"threads_started":1}
I20260811 09:08:29.893224 9513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:29.864449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44720 (local address 127.28.154.190:42507)
0811 09:08:29.864619 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:29.864625 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:29.876101 (+ 11476us) server_negotiation.cc:407] Connection header received
0811 09:08:29.876577 (+ 476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:29.876584 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:29.876711 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:29.876784 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:29.878642 (+ 1858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:29.879819 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:29.882576 (+ 2757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:29.883464 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:29.883579 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:29.886388 (+ 2809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:29.886395 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:29.886400 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:08:29.886675 (+ 275us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:29.890153 (+ 3478us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:29.890920 (+ 767us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:29.890926 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:29.890929 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:29.891087 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:29.891550 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:29.891556 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:29.891560 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:29.891913 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:29.892052 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:29.892664 (+ 612us) server_negotiation.cc:299] Negotiation successful
0811 09:08:29.892903 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 09:08:29.894949 9551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:29.864492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:42507 (local address 127.0.0.1:44720)
0811 09:08:29.875955 (+ 11463us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:29.875977 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:29.876404 (+ 427us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:29.876986 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:29.876990 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:29.877816 (+ 826us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:08:29.878466 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:29.878477 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:29.879945 (+ 1468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:29.879951 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:29.882395 (+ 2444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:29.882407 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:29.883875 (+ 1468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:29.883881 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:29.884058 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:29.884738 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:08:29.884759 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:08:29.886157 (+ 1398us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:08:29.890325 (+ 4168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:08:29.890329 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:08:29.890336 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:08:29.890790 (+ 454us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:08:29.891232 (+ 442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:08:29.891237 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:08:29.891241 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:08:29.891399 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:08:29.892129 (+ 730us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:08:29.892133 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:08:29.892354 (+ 221us) client_negotiation.cc:769] Sending connection context
0811 09:08:29.893987 (+ 1633us) client_negotiation.cc:240] Negotiation successful
0811 09:08:29.894371 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11289,"mutex_wait_us":79,"thread_start_us":11201,"threads_started":1}
WARNING: no policy specified for oryx/127.28.154.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.28.154.188@KRBTEST.COM" created.
Aug 11 09:08:29 dist-test-slave-xt4k krb5kdc[8413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439307, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.154.190@KRBTEST.COM for oryx/127.28.154.189@KRBTEST.COM
I20260811 09:08:29.918004 9509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:29.901428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36056 (local address 127.28.154.189:33355)
0811 09:08:29.901679 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:29.901687 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:29.905059 (+ 3372us) server_negotiation.cc:407] Connection header received
0811 09:08:29.905415 (+ 356us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:29.905420 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:29.905511 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:29.905606 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:29.906727 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:29.907454 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:29.908760 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:29.909277 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:29.909391 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:29.912530 (+ 3139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:29.912535 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:29.912539 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:08:29.912571 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:29.915271 (+ 2700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:29.915952 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:29.915956 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:29.915959 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:29.916092 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:29.916532 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:29.916536 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:29.916539 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:29.916793 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:29.916930 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:29.917442 (+ 512us) server_negotiation.cc:299] Negotiation successful
0811 09:08:29.917658 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129}
I20260811 09:08:29.918071 9553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:29.901236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.189:33355 (local address 127.0.0.1:36056)
0811 09:08:29.904917 (+ 3681us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:29.904954 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:29.905212 (+ 258us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:29.905681 (+ 469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:29.905688 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:29.906340 (+ 652us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:08:29.906569 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:29.906580 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:29.907581 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:29.907585 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:29.908626 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:29.908641 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:29.909442 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:29.909448 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:29.909631 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:29.910223 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:08:29.910240 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:08:29.912326 (+ 2086us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:08:29.915400 (+ 3074us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:08:29.915414 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:08:29.915423 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:08:29.915829 (+ 406us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:08:29.916196 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:08:29.916202 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:08:29.916206 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:08:29.916411 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:08:29.916924 (+ 513us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:08:29.916933 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:08:29.917115 (+ 182us) client_negotiation.cc:769] Sending connection context
0811 09:08:29.917291 (+ 176us) client_negotiation.cc:240] Negotiation successful
0811 09:08:29.917631 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3450,"thread_start_us":3266,"threads_started":1}
I20260811 09:08:29.920217 8495 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.28.154.189:33355: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "169951c4d2524ade9f73f60b2d109e3a"
member_type: NON_VOTER
last_known_addr {
host: "127.28.154.189"
port: 33355
}
attrs {
promote: true
}
}
dest_uuid: "41f9d60d712e4276b0da79549a2090e5"
cas_config_opid_index: -1
I20260811 09:08:29.921653 8495 consensus_queue.cc:237] T 00000000000000000000000000000000 P 41f9d60d712e4276b0da79549a2090e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "41f9d60d712e4276b0da79549a2090e5" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 42507 } } peers { permanent_uuid: "169951c4d2524ade9f73f60b2d109e3a" member_type: NON_VOTER last_known_addr { host: "127.28.154.189" port: 33355 } attrs { promote: true } }
I20260811 09:08:29.930213 9561 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 41f9d60d712e4276b0da79549a2090e5 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 169951c4d2524ade9f73f60b2d109e3a (127.28.154.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "41f9d60d712e4276b0da79549a2090e5" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 42507 } } peers { permanent_uuid: "169951c4d2524ade9f73f60b2d109e3a" member_type: NON_VOTER last_known_addr { host: "127.28.154.189" port: 33355 } attrs { promote: true } } }
I20260811 09:08:29.931345 9458 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 169951c4d2524ade9f73f60b2d109e3a [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:08:29.934960 9561 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.28.154.189:33355
I20260811 09:08:29.935969 9458 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 169951c4d2524ade9f73f60b2d109e3a [term 1 FOLLOWER]: Refusing update from remote peer 41f9d60d712e4276b0da79549a2090e5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260811 09:08:29.936077 9381 master.cc:537] Master@127.28.154.189:33355 shutting down...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.28.154.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/krb5kdc/oryx_127.28.154.188.keytab.
Entry for principal oryx/127.28.154.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/krb5kdc/oryx_127.28.154.188.keytab.
I20260811 09:08:29.939826 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.154.188:37317
--webserver_interface=127.28.154.188
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/krb5kdc/oryx_127.28.154.188.keytab
--principal=oryx/127.28.154.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:33603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.154.190:42507,127.28.154.189:33355,127.28.154.188:37317
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 09:08:29.953334 9560 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41f9d60d712e4276b0da79549a2090e5 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "41f9d60d712e4276b0da79549a2090e5" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "41f9d60d712e4276b0da79549a2090e5" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 42507 } } peers { permanent_uuid: "169951c4d2524ade9f73f60b2d109e3a" member_type: NON_VOTER last_known_addr { host: "127.28.154.189" port: 33355 } attrs { promote: true } } }
I20260811 09:08:29.953794 9560 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41f9d60d712e4276b0da79549a2090e5 [sys.catalog]: This master's current role is: LEADER
I20260811 09:08:29.954269 9561 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 41f9d60d712e4276b0da79549a2090e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "169951c4d2524ade9f73f60b2d109e3a" member_type: NON_VOTER last_known_addr { host: "127.28.154.189" port: 33355 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:08:29.954366 9381 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 169951c4d2524ade9f73f60b2d109e3a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:08:29.954907 9381 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 169951c4d2524ade9f73f60b2d109e3a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:08:29.955122 9381 tablet_replica.cc:330] T 00000000000000000000000000000000 P 169951c4d2524ade9f73f60b2d109e3a: stopping tablet replica
I20260811 09:08:29.963986 9381 master.cc:552] Master@127.28.154.189:33355 shutdown complete.
I20260811 09:08:29.964138 9381 master_runner.cc:305] Clearing existing system tablet
I20260811 09:08:29.965142 9381 fs_manager.cc:359] Metadata directory not provided
I20260811 09:08:29.965246 9381 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:08:29.965248 9553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:29.964691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.189:33355 (local address 127.0.0.1:36064)
0811 09:08:29.964889 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:29.965027 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.189:33355: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260811 09:08:29.967871 8475 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.28.154.189:33355: connect: Connection refused (error 111)
I20260811 09:08:29.969933 9553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:29.969469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.189:33355 (local address 127.0.0.1:36080)
0811 09:08:29.969661 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:29.969751 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.189:33355: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260811 09:08:29.970117 8475 consensus_peers.cc:487] T 00000000000000000000000000000000 P 41f9d60d712e4276b0da79549a2090e5 -> Peer 169951c4d2524ade9f73f60b2d109e3a (127.28.154.189:33355): Couldn't send request to peer 169951c4d2524ade9f73f60b2d109e3a. Status: Network error: Client connection negotiation failed: client connection to 127.28.154.189:33355: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 09:08:29.970781 9381 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.002s
I20260811 09:08:29.972523 9571 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:08:29.972910 9381 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:08:29.973356 9381 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260811 09:08:29.973492 9381 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/master-1/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/master-1/wal
uuid: "169951c4d2524ade9f73f60b2d109e3a"
format_stamp: "Formatted at 2026-08-11 09:08:29 on dist-test-slave-xt4k"
server_key: "3f8296f511286c54d2d0a33238a8482c"
server_key_iv: "f740f8d7ab94a3489eed101465d39b47"
server_key_version: "encryptionkey@0"
I20260811 09:08:29.979112 9381 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 169951c4d2524ade9f73f60b2d109e3a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:08:29.986738 9381 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 169951c4d2524ade9f73f60b2d109e3a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260811 09:08:29.987082 9381 log.cc:1196] T 00000000000000000000000000000000 P 169951c4d2524ade9f73f60b2d109e3a: Deleting WAL directory at /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260811 09:08:29.987661 9381 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 169951c4d2524ade9f73f60b2d109e3a: Deleting consensus metadata
I20260811 09:08:29.987984 9381 master_runner.cc:315] Copying system tablet from 127.28.154.190:42507
I20260811 09:08:29.991070 9381 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 169951c4d2524ade9f73f60b2d109e3a: tablet copy: Beginning tablet copy session from remote peer at address 127.28.154.190:42507
I20260811 09:08:30.013662 9513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:29.992417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44728 (local address 127.28.154.190:42507)
0811 09:08:29.992624 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:29.992630 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:29.998599 (+ 5969us) server_negotiation.cc:407] Connection header received
0811 09:08:29.998871 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:29.998877 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:29.998973 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:29.999124 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:30.000243 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:30.001530 (+ 1287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:30.003485 (+ 1955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:30.004154 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:30.004295 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:30.006661 (+ 2366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:30.006669 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:30.006673 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:08:30.006713 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:30.010363 (+ 3650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:30.011277 (+ 914us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:30.011283 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:30.011286 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:30.011468 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:30.011916 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:30.011921 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:30.011924 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:30.012320 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:30.012505 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:30.013039 (+ 534us) server_negotiation.cc:299] Negotiation successful
0811 09:08:30.013285 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 09:08:30.013674 9583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:29.992316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:42507 (local address 127.0.0.1:44728)
0811 09:08:29.998415 (+ 6099us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:29.998445 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:29.998654 (+ 209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:29.999160 (+ 506us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:29.999165 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:29.999737 (+ 572us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:08:30.000060 (+ 323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:30.000074 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:30.001697 (+ 1623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:30.001704 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:30.003294 (+ 1590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:30.003314 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:30.004380 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:30.004388 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:30.004585 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:30.005219 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:08:30.005243 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:08:30.006408 (+ 1165us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:08:30.010549 (+ 4141us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:08:30.010555 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:08:30.010560 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:08:30.011119 (+ 559us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:08:30.011589 (+ 470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:08:30.011595 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:08:30.011599 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:08:30.011794 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:08:30.012487 (+ 693us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:08:30.012492 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:08:30.012682 (+ 190us) client_negotiation.cc:769] Sending connection context
0811 09:08:30.012860 (+ 178us) client_negotiation.cc:240] Negotiation successful
0811 09:08:30.013216 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5912,"thread_start_us":5742,"threads_started":1}
I20260811 09:08:30.015269 8529 tablet_copy_service.cc:140] P 41f9d60d712e4276b0da79549a2090e5: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 169951c4d2524ade9f73f60b2d109e3a ({username='oryx', principal='oryx/127.28.154.189@KRBTEST.COM'} at 127.0.0.1:44728)
I20260811 09:08:30.015801 8529 tablet_copy_service.cc:161] P 41f9d60d712e4276b0da79549a2090e5: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 169951c4d2524ade9f73f60b2d109e3a at {username='oryx', principal='oryx/127.28.154.189@KRBTEST.COM'} at 127.0.0.1:44728: session id = 169951c4d2524ade9f73f60b2d109e3a-00000000000000000000000000000000
I20260811 09:08:30.021410 8529 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 41f9d60d712e4276b0da79549a2090e5: Tablet Copy: opened 0 blocks and 1 log segments
I20260811 09:08:30.024721 9381 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260811 09:08:30.031944 9381 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 169951c4d2524ade9f73f60b2d109e3a: tablet copy: Starting download of 0 data blocks...
I20260811 09:08:30.032440 9381 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 169951c4d2524ade9f73f60b2d109e3a: tablet copy: Starting download of 1 WAL segments...
I20260811 09:08:30.036790 9381 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 169951c4d2524ade9f73f60b2d109e3a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260811 09:08:30.042212 8529 tablet_copy_service.cc:342] P 41f9d60d712e4276b0da79549a2090e5: Request end of tablet copy session 169951c4d2524ade9f73f60b2d109e3a-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.28.154.189@KRBTEST.COM'} at 127.0.0.1:44728
I20260811 09:08:30.042550 8529 tablet_copy_service.cc:433] P 41f9d60d712e4276b0da79549a2090e5: ending tablet copy session 169951c4d2524ade9f73f60b2d109e3a-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 169951c4d2524ade9f73f60b2d109e3a
I20260811 09:08:30.055675 9381 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:08:30.058650 9585 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:08:30.065255 9590 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:08:30.072628 9381 server_base.cc:1034] running on GCE node
W20260811 09:08:30.073797 9592 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:08:30.074324 9381 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:08:30.075065 9381 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:08:30.076315 9381 hybrid_clock.cc:648] HybridClock initialized: now 1786439310076251 us; error 28 us; skew 500 ppm
I20260811 09:08:30.076583 9381 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:08:30 dist-test-slave-xt4k krb5kdc[8413](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439310, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.154.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:08:30.078846 9381 init.cc:377] Logged in from keytab as oryx/127.28.154.189@KRBTEST.COM (short username oryx)
I20260811 09:08:30.080380 9381 webserver.cc:460] Webserver started at http://127.28.154.189:41719/ using document root <none> and password file <none>
I20260811 09:08:30.080827 9381 fs_manager.cc:359] Metadata directory not provided
I20260811 09:08:30.080911 9381 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:08:30.084444 9381 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.004s
I20260811 09:08:30.086673 9603 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:08:30.087262 9381 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260811 09:08:30.087419 9381 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/master-1/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/master-1/wal
uuid: "169951c4d2524ade9f73f60b2d109e3a"
format_stamp: "Formatted at 2026-08-11 09:08:29 on dist-test-slave-xt4k"
server_key: "3f8296f511286c54d2d0a33238a8482c"
server_key_iv: "f740f8d7ab94a3489eed101465d39b47"
server_key_version: "encryptionkey@0"
I20260811 09:08:30.087604 9381 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:08:30.116142 9381 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:08:30.120699 9381 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:08:30.149260 9381 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.189:33355
I20260811 09:08:30.149317 9701 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.189:33355 every 8 connection(s)
I20260811 09:08:30.150352 9381 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/master-1/data/info.pb
I20260811 09:08:30.155918 9702 sys_catalog.cc:263] Verifying existing consensus state
I20260811 09:08:30.160470 9702 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 169951c4d2524ade9f73f60b2d109e3a: Bootstrap starting.
I20260811 09:08:30.182850 9702 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 169951c4d2524ade9f73f60b2d109e3a: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 09:08:30.183954 9702 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 169951c4d2524ade9f73f60b2d109e3a: Bootstrap complete.
I20260811 09:08:30.185252 9702 raft_consensus.cc:348] T 00000000000000000000000000000000 P 169951c4d2524ade9f73f60b2d109e3a [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "41f9d60d712e4276b0da79549a2090e5" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 42507 } } peers { permanent_uuid: "169951c4d2524ade9f73f60b2d109e3a" member_type: NON_VOTER last_known_addr { host: "127.28.154.189" port: 33355 } attrs { promote: true } }
I20260811 09:08:30.185555 9702 raft_consensus.cc:729] T 00000000000000000000000000000000 P 169951c4d2524ade9f73f60b2d109e3a [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 169951c4d2524ade9f73f60b2d109e3a, State: Initialized, Role: LEARNER
I20260811 09:08:30.185833 9702 consensus_queue.cc:260] T 00000000000000000000000000000000 P 169951c4d2524ade9f73f60b2d109e3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "41f9d60d712e4276b0da79549a2090e5" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 42507 } } peers { permanent_uuid: "169951c4d2524ade9f73f60b2d109e3a" member_type: NON_VOTER last_known_addr { host: "127.28.154.189" port: 33355 } attrs { promote: true } }
I20260811 09:08:30.187213 9702 sys_catalog.cc:564] T 00000000000000000000000000000000 P 169951c4d2524ade9f73f60b2d109e3a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:08:30.186877 9707 sys_catalog.cc:455] T 00000000000000000000000000000000 P 169951c4d2524ade9f73f60b2d109e3a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "41f9d60d712e4276b0da79549a2090e5" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 42507 } } peers { permanent_uuid: "169951c4d2524ade9f73f60b2d109e3a" member_type: NON_VOTER last_known_addr { host: "127.28.154.189" port: 33355 } attrs { promote: true } } }
I20260811 09:08:30.187527 9707 sys_catalog.cc:458] T 00000000000000000000000000000000 P 169951c4d2524ade9f73f60b2d109e3a [sys.catalog]: This master's current role is: LEARNER
I20260811 09:08:30.201130 9727 catalog_manager.cc:1259] Loaded cluster ID: 039df53890a0413897b986c5c7822102
I20260811 09:08:30.201349 9727 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 169951c4d2524ade9f73f60b2d109e3a: loading cluster ID for follower catalog manager: success
I20260811 09:08:30.205071 9727 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 169951c4d2524ade9f73f60b2d109e3a: acquiring CA information for follower catalog manager: success
I20260811 09:08:30.207839 9727 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 169951c4d2524ade9f73f60b2d109e3a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 09:08:30.261291 9367 heartbeater.cc:503] Master 127.28.154.190:42507 was elected leader, sending a full tablet report...
W20260811 09:08:30.360014 9566 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:08:30.360378 9566 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:08:30.360538 9566 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:08:30.368687 9566 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:08:30.368808 9566 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:08:30.368892 9566 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:08:30.368980 9566 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:08:30.369052 9566 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:08:30.381505 9566 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:33603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.28.154.190:42507,127.28.154.189:33355,127.28.154.188:37317
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/krb5kdc/oryx_127.28.154.188.keytab
--principal=oryx/127.28.154.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.154.188:37317
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.28.154.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:08:30.382961 9566 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:08:30.384497 9566 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:08:30.400899 9736 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:08:30.401727 9737 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:08:30.402522 9566 server_base.cc:1034] running on GCE node
W20260811 09:08:30.402626 9741 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:08:30.403606 9566 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:08:30.405284 9566 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:08:30.406508 9566 hybrid_clock.cc:648] HybridClock initialized: now 1786439310406455 us; error 44 us; skew 500 ppm
I20260811 09:08:30.406929 9566 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:08:30 dist-test-slave-xt4k krb5kdc[8413](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439310, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.154.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:08:30.410449 9566 init.cc:377] Logged in from keytab as oryx/127.28.154.188@KRBTEST.COM (short username oryx)
I20260811 09:08:30.412109 9566 webserver.cc:460] Webserver started at http://127.28.154.188:42727/ using document root <none> and password file <none>
I20260811 09:08:30.412901 9566 fs_manager.cc:359] Metadata directory not provided
I20260811 09:08:30.413030 9566 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:08:30.413409 9566 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:08:30.416846 9566 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/master-2/data/instance:
uuid: "c6402a22cf5b441bbde21194d27f3d49"
format_stamp: "Formatted at 2026-08-11 09:08:30 on dist-test-slave-xt4k"
server_key: "18f654dfd0aef5f4b602be95882dd86d"
server_key_iv: "f9407f5eb909185f30692ad2822ce492"
server_key_version: "encryptionkey@0"
I20260811 09:08:30.417821 9566 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/master-2/wal/instance:
uuid: "c6402a22cf5b441bbde21194d27f3d49"
format_stamp: "Formatted at 2026-08-11 09:08:30 on dist-test-slave-xt4k"
server_key: "18f654dfd0aef5f4b602be95882dd86d"
server_key_iv: "f9407f5eb909185f30692ad2822ce492"
server_key_version: "encryptionkey@0"
I20260811 09:08:30.424531 9566 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260811 09:08:30.428596 9755 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:08:30.430687 9566 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 09:08:30.430869 9566 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/master-2/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/master-2/wal
uuid: "c6402a22cf5b441bbde21194d27f3d49"
format_stamp: "Formatted at 2026-08-11 09:08:30 on dist-test-slave-xt4k"
server_key: "18f654dfd0aef5f4b602be95882dd86d"
server_key_iv: "f9407f5eb909185f30692ad2822ce492"
server_key_version: "encryptionkey@0"
I20260811 09:08:30.431080 9566 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:08:30.455013 9566 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:08:30.460520 9566 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:08:30.460808 9566 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:08:30.484193 9566 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.188:37317
I20260811 09:08:30.484259 9852 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.188:37317 every 8 connection(s)
I20260811 09:08:30.485668 9566 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/master-2/data/info.pb
I20260811 09:08:30.489755 9853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:08:30.495772 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 9566
I20260811 09:08:30.497139 9853 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.154.190" port: 42507 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 09:08:30 dist-test-slave-xt4k krb5kdc[8413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439310, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.154.188@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM
Aug 11 09:08:30 dist-test-slave-xt4k krb5kdc[8413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439306, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.28.154.188@KRBTEST.COM
I20260811 09:08:30.519568 9513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:30.500822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44736 (local address 127.28.154.190:42507)
0811 09:08:30.501033 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:30.501040 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:30.501780 (+ 740us) server_negotiation.cc:407] Connection header received
0811 09:08:30.502218 (+ 438us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:30.502224 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:30.502300 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:30.502429 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:30.504253 (+ 1824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:30.505460 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:30.507694 (+ 2234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:30.508295 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:30.508446 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:30.512747 (+ 4301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:30.512754 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:30.512759 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:08:30.512800 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:30.515936 (+ 3136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:30.516828 (+ 892us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:30.516834 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:30.516837 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:30.517023 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:30.517523 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:30.517528 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:30.517530 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:30.517913 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:30.518048 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:30.518953 (+ 905us) server_negotiation.cc:299] Negotiation successful
0811 09:08:30.519188 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 09:08:30.521193 9859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:30.500660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:42507 (local address 127.0.0.1:44736)
0811 09:08:30.501622 (+ 962us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:30.501667 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:30.502004 (+ 337us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:30.502524 (+ 520us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:30.502530 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:30.503476 (+ 946us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:08:30.504059 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:30.504087 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:30.505641 (+ 1554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:30.505645 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:30.507524 (+ 1879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:30.507540 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:30.508591 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:30.508597 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:30.508797 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:30.509605 (+ 808us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:08:30.509642 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:08:30.512553 (+ 2911us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:08:30.516070 (+ 3517us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:08:30.516080 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:08:30.516091 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:08:30.516709 (+ 618us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:08:30.517170 (+ 461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:08:30.517174 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:08:30.517176 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:08:30.517399 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:08:30.518056 (+ 657us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:08:30.518065 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:08:30.518543 (+ 478us) client_negotiation.cc:769] Sending connection context
0811 09:08:30.518848 (+ 305us) client_negotiation.cc:240] Negotiation successful
0811 09:08:30.519400 (+ 552us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":686,"thread_start_us":516,"threads_started":1}
I20260811 09:08:30.521240 9858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:30.498245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49370 (local address 127.28.154.188:37317)
0811 09:08:30.499549 (+ 1304us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:30.499567 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:30.499737 (+ 170us) server_negotiation.cc:407] Connection header received
0811 09:08:30.501960 (+ 2223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:30.501983 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:30.502242 (+ 259us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:30.502485 (+ 243us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:30.504402 (+ 1917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:30.506045 (+ 1643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:30.507835 (+ 1790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:30.508735 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:30.508824 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:30.513823 (+ 4999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:30.513835 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:30.513854 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:08:30.513897 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:30.517268 (+ 3371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:30.518096 (+ 828us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:30.518104 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:30.518107 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:30.518260 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:30.518696 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:30.518701 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:30.518703 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:30.519124 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:30.519413 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:30.520023 (+ 610us) server_negotiation.cc:299] Negotiation successful
0811 09:08:30.520267 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":888,"spinlock_wait_cycles":15104,"thread_start_us":618,"threads_started":1}
I20260811 09:08:30.522989 9853 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.154.189" port: 33355 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 09:08:30 dist-test-slave-xt4k krb5kdc[8413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439310, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.154.188@KRBTEST.COM for oryx/127.28.154.189@KRBTEST.COM
I20260811 09:08:30.542754 9859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:30.523940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.189:33355 (local address 127.0.0.1:36092)
0811 09:08:30.524519 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:30.524537 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:30.524630 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:30.529235 (+ 4605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:30.529239 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:30.529677 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:08:30.529844 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:30.529854 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:30.531826 (+ 1972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:30.531831 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:30.532916 (+ 1085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:30.532932 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:30.533903 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:30.533908 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:30.534059 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:30.534852 (+ 793us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:08:30.534870 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:08:30.537157 (+ 2287us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:08:30.540207 (+ 3050us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:08:30.540211 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:08:30.540214 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:08:30.540742 (+ 528us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:08:30.541178 (+ 436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:08:30.541182 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:08:30.541184 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:08:30.541339 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:08:30.541897 (+ 558us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:08:30.541901 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:08:30.542041 (+ 140us) client_negotiation.cc:769] Sending connection context
0811 09:08:30.542167 (+ 126us) client_negotiation.cc:240] Negotiation successful
0811 09:08:30.542469 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":439}
I20260811 09:08:30.542819 9860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:30.523891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36092 (local address 127.28.154.189:33355)
0811 09:08:30.528886 (+ 4995us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:30.528893 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:30.528913 (+ 20us) server_negotiation.cc:407] Connection header received
0811 09:08:30.528973 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:30.528979 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:30.529054 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:30.529162 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:30.529995 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:30.531661 (+ 1666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:30.533063 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:30.533721 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:30.533864 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:30.537418 (+ 3554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:30.537424 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:30.537428 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:08:30.537464 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:30.540077 (+ 2613us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:30.540879 (+ 802us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:30.540884 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:30.540886 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:30.541065 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:30.541424 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:30.541428 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:30.541430 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:30.541767 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:30.541897 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:30.542317 (+ 420us) server_negotiation.cc:299] Negotiation successful
0811 09:08:30.542542 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4832,"thread_start_us":4672,"threads_started":1}
I20260811 09:08:30.544081 9853 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.154.188" port: 37317 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 09:08:30 dist-test-slave-xt4k krb5kdc[8413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439310, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.154.188@KRBTEST.COM for oryx/127.28.154.188@KRBTEST.COM
I20260811 09:08:30.556620 9859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:30.544802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.188:37317 (local address 127.0.0.1:49376)
0811 09:08:30.544960 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:30.544978 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:30.545089 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:30.545319 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:30.545322 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:30.545667 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:08:30.545819 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:30.545826 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:30.546821 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:30.546825 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:30.547736 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:30.547747 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:30.548471 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:30.548475 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:30.548724 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:30.549259 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:08:30.549276 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:08:30.551366 (+ 2090us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:08:30.554326 (+ 2960us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:08:30.554330 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:08:30.554333 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:08:30.554745 (+ 412us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:08:30.555109 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:08:30.555112 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:08:30.555114 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:08:30.555282 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:08:30.555762 (+ 480us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:08:30.555766 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:08:30.555921 (+ 155us) client_negotiation.cc:769] Sending connection context
0811 09:08:30.556037 (+ 116us) client_negotiation.cc:240] Negotiation successful
0811 09:08:30.556229 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260811 09:08:30.556654 9858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:30.544804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49376 (local address 127.28.154.188:37317)
0811 09:08:30.544946 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:30.544950 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:30.545022 (+ 72us) server_negotiation.cc:407] Connection header received
0811 09:08:30.545169 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:30.545172 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:30.545225 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:30.545297 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:30.545972 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:30.546692 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:30.547865 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:30.548292 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:30.548465 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:30.551566 (+ 3101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:30.551571 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:30.551574 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:08:30.551604 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:30.554195 (+ 2591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:30.554853 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:30.554857 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:30.554860 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:30.555017 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:30.555379 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:30.555382 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:30.555384 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:30.555658 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:30.555765 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:30.556176 (+ 411us) server_negotiation.cc:299] Negotiation successful
0811 09:08:30.556403 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 09:08:30.561765 9853 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c6402a22cf5b441bbde21194d27f3d49: Bootstrap starting.
I20260811 09:08:30.565243 9853 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c6402a22cf5b441bbde21194d27f3d49: Neither blocks nor log segments found. Creating new log.
I20260811 09:08:30.566421 9853 log.cc:824] T 00000000000000000000000000000000 P c6402a22cf5b441bbde21194d27f3d49: Log is configured to *not* fsync() on all Append() calls
I20260811 09:08:30.569376 9853 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c6402a22cf5b441bbde21194d27f3d49: No bootstrap required, opened a new log
I20260811 09:08:30.579018 9853 raft_consensus.cc:348] T 00000000000000000000000000000000 P c6402a22cf5b441bbde21194d27f3d49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41f9d60d712e4276b0da79549a2090e5" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 42507 } } peers { permanent_uuid: "169951c4d2524ade9f73f60b2d109e3a" member_type: VOTER last_known_addr { host: "127.28.154.189" port: 33355 } } peers { permanent_uuid: "c6402a22cf5b441bbde21194d27f3d49" member_type: VOTER last_known_addr { host: "127.28.154.188" port: 37317 } }
I20260811 09:08:30.579375 9853 raft_consensus.cc:374] T 00000000000000000000000000000000 P c6402a22cf5b441bbde21194d27f3d49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:08:30.579507 9853 raft_consensus.cc:729] T 00000000000000000000000000000000 P c6402a22cf5b441bbde21194d27f3d49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6402a22cf5b441bbde21194d27f3d49, State: Initialized, Role: FOLLOWER
I20260811 09:08:30.580118 9853 consensus_queue.cc:260] T 00000000000000000000000000000000 P c6402a22cf5b441bbde21194d27f3d49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41f9d60d712e4276b0da79549a2090e5" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 42507 } } peers { permanent_uuid: "169951c4d2524ade9f73f60b2d109e3a" member_type: VOTER last_known_addr { host: "127.28.154.189" port: 33355 } } peers { permanent_uuid: "c6402a22cf5b441bbde21194d27f3d49" member_type: VOTER last_known_addr { host: "127.28.154.188" port: 37317 } }
I20260811 09:08:30.581141 9867 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6402a22cf5b441bbde21194d27f3d49 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41f9d60d712e4276b0da79549a2090e5" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 42507 } } peers { permanent_uuid: "169951c4d2524ade9f73f60b2d109e3a" member_type: VOTER last_known_addr { host: "127.28.154.189" port: 33355 } } peers { permanent_uuid: "c6402a22cf5b441bbde21194d27f3d49" member_type: VOTER last_known_addr { host: "127.28.154.188" port: 37317 } } }
I20260811 09:08:30.581493 9867 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6402a22cf5b441bbde21194d27f3d49 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 09:08:30.582427 9853 sys_catalog.cc:564] T 00000000000000000000000000000000 P c6402a22cf5b441bbde21194d27f3d49 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:08:30.583385 9865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:30.576854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.189:33355 (local address 127.0.0.1:36104)
0811 09:08:30.577335 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:30.577355 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:30.577486 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:30.577897 (+ 411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:30.577902 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:30.577928 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:08:30.578130 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:30.578139 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:30.579178 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:30.579192 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:30.580956 (+ 1764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:30.580973 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:30.582743 (+ 1770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:30.582748 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:30.582876 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:30.582958 (+ 82us) client_negotiation.cc:769] Sending connection context
0811 09:08:30.583033 (+ 75us) client_negotiation.cc:240] Negotiation successful
0811 09:08:30.583135 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":226,"threads_started":1}
I20260811 09:08:30.583498 9860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:30.577133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36104 (local address 127.28.154.189:33355)
0811 09:08:30.577339 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:30.577344 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:30.577431 (+ 87us) server_negotiation.cc:407] Connection header received
0811 09:08:30.577618 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:30.577622 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:30.577737 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:30.577836 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:08:30.578260 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:30.579046 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:30.581124 (+ 2078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:30.582545 (+ 1421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:30.582731 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:30.582943 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:30.583083 (+ 140us) server_negotiation.cc:299] Negotiation successful
0811 09:08:30.583235 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260811 09:08:30.584722 9656 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 169951c4d2524ade9f73f60b2d109e3a [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.5] Dedup: 1.5->[]
I20260811 09:08:30.585325 9707 sys_catalog.cc:455] T 00000000000000000000000000000000 P 169951c4d2524ade9f73f60b2d109e3a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 41f9d60d712e4276b0da79549a2090e5. Latest consensus state: current_term: 1 leader_uuid: "41f9d60d712e4276b0da79549a2090e5" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "41f9d60d712e4276b0da79549a2090e5" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 42507 } } peers { permanent_uuid: "169951c4d2524ade9f73f60b2d109e3a" member_type: NON_VOTER last_known_addr { host: "127.28.154.189" port: 33355 } attrs { promote: true } } }
I20260811 09:08:30.585585 9707 sys_catalog.cc:458] T 00000000000000000000000000000000 P 169951c4d2524ade9f73f60b2d109e3a [sys.catalog]: This master's current role is: LEARNER
W20260811 09:08:30.593041 9886 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c6402a22cf5b441bbde21194d27f3d49: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 09:08:30.593201 9886 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 09:08:30.597203 9566 master_runner.cc:428] Detected that this master c6402a22cf5b441bbde21194d27f3d49 is joining an existing cluster
I20260811 09:08:30.597343 9566 master_runner.cc:432] Initiating AddMaster RPC to add 127.28.154.188:37317
I20260811 09:08:30.622924 9894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:30.602411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:42507 (local address 127.0.0.1:44742)
0811 09:08:30.605836 (+ 3425us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:30.605855 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:30.606019 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:30.606539 (+ 520us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:30.606544 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:30.607370 (+ 826us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:08:30.608185 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:30.608200 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:30.609788 (+ 1588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:30.609793 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:30.611854 (+ 2061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:30.611872 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:30.613178 (+ 1306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:30.613184 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:30.613379 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:30.614187 (+ 808us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:08:30.614211 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:08:30.615534 (+ 1323us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:08:30.619460 (+ 3926us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:08:30.619464 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:08:30.619468 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:08:30.620011 (+ 543us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:08:30.620524 (+ 513us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:08:30.620529 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:08:30.620533 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:08:30.620724 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:08:30.621453 (+ 729us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:08:30.621459 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:08:30.621755 (+ 296us) client_negotiation.cc:769] Sending connection context
0811 09:08:30.621908 (+ 153us) client_negotiation.cc:240] Negotiation successful
0811 09:08:30.622301 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3178,"mutex_wait_us":4,"thread_start_us":2433,"threads_started":1}
I20260811 09:08:30.623677 9860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:30.602049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36110 (local address 127.28.154.189:33355)
0811 09:08:30.602290 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:30.602297 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:30.606601 (+ 4304us) server_negotiation.cc:407] Connection header received
0811 09:08:30.606852 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:30.606858 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:30.606938 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:30.607002 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:30.609299 (+ 2297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:30.610345 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:30.612938 (+ 2593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:30.613660 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:30.613770 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:30.616419 (+ 2649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:30.616426 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:30.616431 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:08:30.616484 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:30.620056 (+ 3572us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:30.620869 (+ 813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:30.620875 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:30.620878 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:30.621051 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:30.621543 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:30.621548 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:30.621551 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:30.621923 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:30.622043 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:30.622745 (+ 702us) server_negotiation.cc:299] Negotiation successful
0811 09:08:30.622987 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127}
I20260811 09:08:30.624008 9513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:30.602107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44742 (local address 127.28.154.190:42507)
0811 09:08:30.602290 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:30.602297 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:30.605954 (+ 3657us) server_negotiation.cc:407] Connection header received
0811 09:08:30.606139 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:30.606143 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:30.606209 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:30.606315 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:30.608396 (+ 2081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:30.609574 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:30.612057 (+ 2483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:30.612767 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:30.612894 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:30.615770 (+ 2876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:30.615777 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:30.615782 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:08:30.615835 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:30.619299 (+ 3464us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:30.620170 (+ 871us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:30.620176 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:30.620180 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:30.620385 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:30.620867 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:30.620872 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:30.620876 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:30.621251 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:30.621381 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:30.623406 (+ 2025us) server_negotiation.cc:299] Negotiation successful
0811 09:08:30.623652 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 09:08:30.625630 9858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:30.602957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49392 (local address 127.28.154.188:37317)
0811 09:08:30.603150 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:30.603157 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:30.606730 (+ 3573us) server_negotiation.cc:407] Connection header received
0811 09:08:30.606934 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:30.606939 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:30.606999 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:30.607064 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:30.609209 (+ 2145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:30.610379 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:30.612858 (+ 2479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:30.613715 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:30.614181 (+ 466us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:30.617448 (+ 3267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:30.617454 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:30.617458 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:08:30.617491 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:30.621365 (+ 3874us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:30.622235 (+ 870us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:30.622240 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:30.622243 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:30.622478 (+ 235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:30.622916 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:30.622920 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:30.622924 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:30.623552 (+ 628us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:30.624015 (+ 463us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:30.624993 (+ 978us) server_negotiation.cc:299] Negotiation successful
0811 09:08:30.625207 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57,"spinlock_wait_cycles":1792}
I20260811 09:08:30.625233 9895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:30.602594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.189:33355 (local address 127.0.0.1:36110)
0811 09:08:30.606498 (+ 3904us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:30.606523 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:30.606702 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:30.607220 (+ 518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:30.607224 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:30.608540 (+ 1316us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:08:30.609126 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:30.609140 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:30.610517 (+ 1377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:30.610526 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:30.612762 (+ 2236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:30.612776 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:30.614057 (+ 1281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:30.614064 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:30.614257 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:30.614873 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:08:30.614894 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:08:30.616169 (+ 1275us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:08:30.620201 (+ 4032us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:08:30.620205 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:08:30.620208 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:08:30.620746 (+ 538us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:08:30.621201 (+ 455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:08:30.621206 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:08:30.621210 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:08:30.621391 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:08:30.622123 (+ 732us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:08:30.622127 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:08:30.622385 (+ 258us) client_negotiation.cc:769] Sending connection context
0811 09:08:30.624412 (+ 2027us) client_negotiation.cc:240] Negotiation successful
0811 09:08:30.624751 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3768,"mutex_wait_us":122,"spinlock_wait_cycles":77056,"thread_start_us":2944,"threads_started":1}
I20260811 09:08:30.628118 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 8614
I20260811 09:08:30.626271 9899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:30.605954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.188:37317 (local address 127.0.0.1:49392)
0811 09:08:30.606658 (+ 704us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:30.606672 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:30.606823 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:30.607224 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:30.607228 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:30.608525 (+ 1297us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:08:30.609051 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:30.609066 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:30.610684 (+ 1618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:30.610689 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:30.612717 (+ 2028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:30.612733 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:30.614773 (+ 2040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:30.614783 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:30.615037 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:30.615687 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:08:30.615742 (+ 55us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:08:30.617266 (+ 1524us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:08:30.621549 (+ 4283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:08:30.621553 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:08:30.621557 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:08:30.622113 (+ 556us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:08:30.622618 (+ 505us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:08:30.622635 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:08:30.622639 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:08:30.622795 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:08:30.624086 (+ 1291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:08:30.624091 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:08:30.624655 (+ 564us) client_negotiation.cc:769] Sending connection context
0811 09:08:30.625265 (+ 610us) client_negotiation.cc:240] Negotiation successful
0811 09:08:30.625550 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":565,"mutex_wait_us":136,"thread_start_us":1491,"threads_started":1}
Aug 11 09:08:30 dist-test-slave-xt4k krb5kdc[8413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439307, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.154.190@KRBTEST.COM for oryx/127.28.154.188@KRBTEST.COM
Aug 11 09:08:30 dist-test-slave-xt4k krb5kdc[8413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439306, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.28.154.129@KRBTEST.COM
I20260811 09:08:30.646317 9865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:30.631103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.188:37317 (local address 127.0.0.1:49400)
0811 09:08:30.631265 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:30.631284 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:30.631383 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:30.631975 (+ 592us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:30.631980 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:30.632460 (+ 480us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:08:30.632680 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:30.632692 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:30.634167 (+ 1475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:30.634173 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:30.635176 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:30.635190 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:30.636086 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:30.636092 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:30.636288 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:30.636927 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:08:30.636945 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:08:30.639702 (+ 2757us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:08:30.643587 (+ 3885us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:08:30.643594 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:08:30.643598 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:08:30.644098 (+ 500us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:08:30.644508 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:08:30.644513 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:08:30.644517 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:08:30.644676 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:08:30.645294 (+ 618us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:08:30.645299 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:08:30.645470 (+ 171us) client_negotiation.cc:769] Sending connection context
0811 09:08:30.645608 (+ 138us) client_negotiation.cc:240] Negotiation successful
0811 09:08:30.645916 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260811 09:08:30.646349 9858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:30.631449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49400 (local address 127.28.154.188:37317)
0811 09:08:30.631589 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:30.631594 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:30.631611 (+ 17us) server_negotiation.cc:407] Connection header received
0811 09:08:30.631694 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:30.631698 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:30.631825 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:30.631915 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:30.632887 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:30.634009 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:30.635313 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:30.635929 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:30.636042 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:30.639927 (+ 3885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:30.639934 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:30.639939 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:08:30.639979 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:30.643437 (+ 3458us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:30.644218 (+ 781us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:30.644224 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:30.644228 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:30.644400 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:30.644782 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:30.644787 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:30.644790 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:30.645177 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:30.645319 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:30.645734 (+ 415us) server_negotiation.cc:299] Negotiation successful
0811 09:08:30.645980 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260811 09:08:30.647786 8495 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.28.154.188:37317: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "c6402a22cf5b441bbde21194d27f3d49"
member_type: NON_VOTER
last_known_addr {
host: "127.28.154.188"
port: 37317
}
attrs {
promote: true
}
}
dest_uuid: "41f9d60d712e4276b0da79549a2090e5"
cas_config_opid_index: 5
I20260811 09:08:30.648854 8495 consensus_queue.cc:237] T 00000000000000000000000000000000 P 41f9d60d712e4276b0da79549a2090e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "41f9d60d712e4276b0da79549a2090e5" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 42507 } } peers { permanent_uuid: "169951c4d2524ade9f73f60b2d109e3a" member_type: NON_VOTER last_known_addr { host: "127.28.154.189" port: 33355 } attrs { promote: true } } peers { permanent_uuid: "c6402a22cf5b441bbde21194d27f3d49" member_type: NON_VOTER last_known_addr { host: "127.28.154.188" port: 37317 } attrs { promote: true } }
I20260811 09:08:30.650565 9864 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 41f9d60d712e4276b0da79549a2090e5 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER c6402a22cf5b441bbde21194d27f3d49 (127.28.154.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "41f9d60d712e4276b0da79549a2090e5" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 42507 } } peers { permanent_uuid: "169951c4d2524ade9f73f60b2d109e3a" member_type: NON_VOTER last_known_addr { host: "127.28.154.189" port: 33355 } attrs { promote: true } } peers { permanent_uuid: "c6402a22cf5b441bbde21194d27f3d49" member_type: NON_VOTER last_known_addr { host: "127.28.154.188" port: 37317 } attrs { promote: true } } }
I20260811 09:08:30.652912 9900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:30.629568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37776 (local address 127.28.154.129:34797)
0811 09:08:30.630183 (+ 615us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:30.630192 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:30.630222 (+ 30us) server_negotiation.cc:407] Connection header received
0811 09:08:30.630288 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:30.630296 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:30.630586 (+ 290us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:30.630759 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:30.632530 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:30.634190 (+ 1660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:30.636185 (+ 1995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:30.637280 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:30.637422 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:30.642686 (+ 5264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:30.642707 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:30.642732 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:08:30.642777 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:30.647509 (+ 4732us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:30.648461 (+ 952us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:30.648471 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:30.648474 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:30.648646 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:30.649183 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:30.649189 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:30.649192 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:30.649754 (+ 562us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:30.650691 (+ 937us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:30.652111 (+ 1420us) server_negotiation.cc:299] Negotiation successful
0811 09:08:30.652444 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":418,"thread_start_us":1962,"threads_started":1}
I20260811 09:08:30.653335 9656 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 169951c4d2524ade9f73f60b2d109e3a [term 1 LEARNER]: Refusing update from remote peer 41f9d60d712e4276b0da79549a2090e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260811 09:08:30.655465 8841 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:08:30.656492 9805 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c6402a22cf5b441bbde21194d27f3d49 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:08:30.660657 9805 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c6402a22cf5b441bbde21194d27f3d49 [term 1 FOLLOWER]: Refusing update from remote peer 41f9d60d712e4276b0da79549a2090e5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260811 09:08:30.667683 9901 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 41f9d60d712e4276b0da79549a2090e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "169951c4d2524ade9f73f60b2d109e3a" member_type: NON_VOTER last_known_addr { host: "127.28.154.189" port: 33355 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260811 09:08:30.670874 9656 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 169951c4d2524ade9f73f60b2d109e3a [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER c6402a22cf5b441bbde21194d27f3d49 (127.28.154.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "41f9d60d712e4276b0da79549a2090e5" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 42507 } } peers { permanent_uuid: "169951c4d2524ade9f73f60b2d109e3a" member_type: NON_VOTER last_known_addr { host: "127.28.154.189" port: 33355 } attrs { promote: true } } peers { permanent_uuid: "c6402a22cf5b441bbde21194d27f3d49" member_type: NON_VOTER last_known_addr { host: "127.28.154.188" port: 37317 } attrs { promote: true } } }
I20260811 09:08:30.673153 9867 raft_consensus.cc:482] T 00000000000000000000000000000000 P c6402a22cf5b441bbde21194d27f3d49 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:08:30.673512 9867 raft_consensus.cc:504] T 00000000000000000000000000000000 P c6402a22cf5b441bbde21194d27f3d49 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41f9d60d712e4276b0da79549a2090e5" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 42507 } } peers { permanent_uuid: "169951c4d2524ade9f73f60b2d109e3a" member_type: VOTER last_known_addr { host: "127.28.154.189" port: 33355 } } peers { permanent_uuid: "c6402a22cf5b441bbde21194d27f3d49" member_type: VOTER last_known_addr { host: "127.28.154.188" port: 37317 } }
I20260811 09:08:30.675199 9867 leader_election.cc:288] T 00000000000000000000000000000000 P c6402a22cf5b441bbde21194d27f3d49 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 41f9d60d712e4276b0da79549a2090e5 (127.28.154.190:42507), 169951c4d2524ade9f73f60b2d109e3a (127.28.154.189:33355)
I20260811 09:08:30.675558 9707 sys_catalog.cc:455] T 00000000000000000000000000000000 P 169951c4d2524ade9f73f60b2d109e3a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "41f9d60d712e4276b0da79549a2090e5" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "41f9d60d712e4276b0da79549a2090e5" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 42507 } } peers { permanent_uuid: "169951c4d2524ade9f73f60b2d109e3a" member_type: NON_VOTER last_known_addr { host: "127.28.154.189" port: 33355 } attrs { promote: true } } peers { permanent_uuid: "c6402a22cf5b441bbde21194d27f3d49" member_type: NON_VOTER last_known_addr { host: "127.28.154.188" port: 37317 } attrs { promote: true } } }
I20260811 09:08:30.675849 9707 sys_catalog.cc:458] T 00000000000000000000000000000000 P 169951c4d2524ade9f73f60b2d109e3a [sys.catalog]: This master's current role is: LEARNER
I20260811 09:08:30.677016 9906 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 41f9d60d712e4276b0da79549a2090e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c6402a22cf5b441bbde21194d27f3d49" member_type: NON_VOTER last_known_addr { host: "127.28.154.188" port: 37317 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:08:30.677367 9656 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c6402a22cf5b441bbde21194d27f3d49" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "169951c4d2524ade9f73f60b2d109e3a" is_pre_election: true
I20260811 09:08:30.679215 8511 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c6402a22cf5b441bbde21194d27f3d49" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41f9d60d712e4276b0da79549a2090e5" is_pre_election: true
I20260811 09:08:30.679903 9864 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.28.154.188:37317
I20260811 09:08:30.680680 9756 leader_election.cc:302] T 00000000000000000000000000000000 P c6402a22cf5b441bbde21194d27f3d49 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c6402a22cf5b441bbde21194d27f3d49; no voters: 169951c4d2524ade9f73f60b2d109e3a, 41f9d60d712e4276b0da79549a2090e5
I20260811 09:08:30.680905 9864 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 41f9d60d712e4276b0da79549a2090e5: attempting to promote NON_VOTER 169951c4d2524ade9f73f60b2d109e3a to VOTER
I20260811 09:08:30.681491 9867 raft_consensus.cc:2726] T 00000000000000000000000000000000 P c6402a22cf5b441bbde21194d27f3d49 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 09:08:30.681731 9864 consensus_queue.cc:237] T 00000000000000000000000000000000 P 41f9d60d712e4276b0da79549a2090e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "41f9d60d712e4276b0da79549a2090e5" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 42507 } } peers { permanent_uuid: "169951c4d2524ade9f73f60b2d109e3a" member_type: VOTER last_known_addr { host: "127.28.154.189" port: 33355 } attrs { promote: false } } peers { permanent_uuid: "c6402a22cf5b441bbde21194d27f3d49" member_type: NON_VOTER last_known_addr { host: "127.28.154.188" port: 37317 } attrs { promote: true } }
I20260811 09:08:30.684267 9566 master.cc:537] Master@127.28.154.188:37317 shutting down...
I20260811 09:08:30.680620 9901 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41f9d60d712e4276b0da79549a2090e5 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "41f9d60d712e4276b0da79549a2090e5" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "41f9d60d712e4276b0da79549a2090e5" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 42507 } } peers { permanent_uuid: "169951c4d2524ade9f73f60b2d109e3a" member_type: NON_VOTER last_known_addr { host: "127.28.154.189" port: 33355 } attrs { promote: true } } peers { permanent_uuid: "c6402a22cf5b441bbde21194d27f3d49" member_type: NON_VOTER last_known_addr { host: "127.28.154.188" port: 37317 } attrs { promote: true } } }
I20260811 09:08:30.685865 9901 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41f9d60d712e4276b0da79549a2090e5 [sys.catalog]: This master's current role is: LEADER
I20260811 09:08:30.703501 9864 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41f9d60d712e4276b0da79549a2090e5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "41f9d60d712e4276b0da79549a2090e5" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "41f9d60d712e4276b0da79549a2090e5" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 42507 } } peers { permanent_uuid: "169951c4d2524ade9f73f60b2d109e3a" member_type: NON_VOTER last_known_addr { host: "127.28.154.189" port: 33355 } attrs { promote: true } } peers { permanent_uuid: "c6402a22cf5b441bbde21194d27f3d49" member_type: NON_VOTER last_known_addr { host: "127.28.154.188" port: 37317 } attrs { promote: true } } } pending_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "41f9d60d712e4276b0da79549a2090e5" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 42507 } } peers { permanent_uuid: "169951c4d2524ade9f73f60b2d109e3a" member_type: VOTER last_known_addr { host: "127.28.154.189" port: 33355 } attrs { promote: false } } peers { permanent_uuid: "c6402a22cf5b441bbde21194d27f3d49" member_type: NON_VOTER last_known_addr { host: "127.28.154.188" port: 37317 } attrs { promote: true } } }
I20260811 09:08:30.703941 9864 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41f9d60d712e4276b0da79549a2090e5 [sys.catalog]: This master's current role is: LEADER
I20260811 09:08:30.704661 9656 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 169951c4d2524ade9f73f60b2d109e3a [term 1 LEARNER]: Refusing update from remote peer 41f9d60d712e4276b0da79549a2090e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260811 09:08:30.705783 9864 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 41f9d60d712e4276b0da79549a2090e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "169951c4d2524ade9f73f60b2d109e3a" member_type: VOTER last_known_addr { host: "127.28.154.189" port: 33355 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260811 09:08:30.709612 9656 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 169951c4d2524ade9f73f60b2d109e3a [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 169951c4d2524ade9f73f60b2d109e3a (127.28.154.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "41f9d60d712e4276b0da79549a2090e5" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 42507 } } peers { permanent_uuid: "169951c4d2524ade9f73f60b2d109e3a" member_type: VOTER last_known_addr { host: "127.28.154.189" port: 33355 } attrs { promote: false } } peers { permanent_uuid: "c6402a22cf5b441bbde21194d27f3d49" member_type: NON_VOTER last_known_addr { host: "127.28.154.188" port: 37317 } attrs { promote: true } } }
W20260811 09:08:30.713300 9765 tablet.cc:2369] T 00000000000000000000000000000000 P c6402a22cf5b441bbde21194d27f3d49: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 09:08:30.714761 9707 sys_catalog.cc:455] T 00000000000000000000000000000000 P 169951c4d2524ade9f73f60b2d109e3a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "41f9d60d712e4276b0da79549a2090e5" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "41f9d60d712e4276b0da79549a2090e5" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 42507 } } peers { permanent_uuid: "169951c4d2524ade9f73f60b2d109e3a" member_type: VOTER last_known_addr { host: "127.28.154.189" port: 33355 } attrs { promote: false } } peers { permanent_uuid: "c6402a22cf5b441bbde21194d27f3d49" member_type: NON_VOTER last_known_addr { host: "127.28.154.188" port: 37317 } attrs { promote: true } } }
I20260811 09:08:30.715055 9707 sys_catalog.cc:458] T 00000000000000000000000000000000 P 169951c4d2524ade9f73f60b2d109e3a [sys.catalog]: This master's current role is: FOLLOWER
W20260811 09:08:30.718695 8473 consensus_peers.cc:487] T 00000000000000000000000000000000 P 41f9d60d712e4276b0da79549a2090e5 -> Peer c6402a22cf5b441bbde21194d27f3d49 (127.28.154.188:37317): Couldn't send request to peer c6402a22cf5b441bbde21194d27f3d49. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260811 09:08:30.717519 9906 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 41f9d60d712e4276b0da79549a2090e5 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 169951c4d2524ade9f73f60b2d109e3a (127.28.154.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "41f9d60d712e4276b0da79549a2090e5" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 42507 } } peers { permanent_uuid: "169951c4d2524ade9f73f60b2d109e3a" member_type: VOTER last_known_addr { host: "127.28.154.189" port: 33355 } attrs { promote: false } } peers { permanent_uuid: "c6402a22cf5b441bbde21194d27f3d49" member_type: NON_VOTER last_known_addr { host: "127.28.154.188" port: 37317 } attrs { promote: true } } }
I20260811 09:08:30.724265 9805 raft_consensus.cc:858] T 00000000000000000000000000000000 P c6402a22cf5b441bbde21194d27f3d49 [term 1 NON_PARTICIPANT]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41f9d60d712e4276b0da79549a2090e5" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 42507 } } peers { permanent_uuid: "169951c4d2524ade9f73f60b2d109e3a" member_type: VOTER last_known_addr { host: "127.28.154.189" port: 33355 } } peers { permanent_uuid: "c6402a22cf5b441bbde21194d27f3d49" member_type: VOTER last_known_addr { host: "127.28.154.188" port: 37317 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "41f9d60d712e4276b0da79549a2090e5" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 42507 } } peers { permanent_uuid: "169951c4d2524ade9f73f60b2d109e3a" member_type: NON_VOTER last_known_addr { host: "127.28.154.189" port: 33355 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "41f9d60d712e4276b0da79549a2090e5" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 42507 } } peers { permanent_uuid: "169951c4d2524ade9f73f60b2d109e3a" member_type: NON_VOTER last_known_addr { host: "127.28.154.189" port: 33355 } attrs { promote: true } } peers { permanent_uuid: "c6402a22cf5b441bbde21194d27f3d49" member_type: NON_VOTER last_known_addr { host: "127.28.154.188" port: 37317 } attrs { promote: true } }
I20260811 09:08:30.724254 9901 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41f9d60d712e4276b0da79549a2090e5 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "41f9d60d712e4276b0da79549a2090e5" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "41f9d60d712e4276b0da79549a2090e5" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 42507 } } peers { permanent_uuid: "169951c4d2524ade9f73f60b2d109e3a" member_type: VOTER last_known_addr { host: "127.28.154.189" port: 33355 } attrs { promote: false } } peers { permanent_uuid: "c6402a22cf5b441bbde21194d27f3d49" member_type: NON_VOTER last_known_addr { host: "127.28.154.188" port: 37317 } attrs { promote: true } } }
I20260811 09:08:30.724586 9901 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41f9d60d712e4276b0da79549a2090e5 [sys.catalog]: This master's current role is: LEADER
W20260811 09:08:30.724498 9805 raft_consensus.cc:1565] T 00000000000000000000000000000000 P c6402a22cf5b441bbde21194d27f3d49 [term 1 NON_PARTICIPANT]: Could not prepare op 'term: 1 index: 6' and following 0 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41f9d60d712e4276b0da79549a2090e5" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 42507 } } peers { permanent_uuid: "169951c4d2524ade9f73f60b2d109e3a" member_type: VOTER last_known_addr { host: "127.28.154.189" port: 33355 } } peers { permanent_uuid: "c6402a22cf5b441bbde21194d27f3d49" member_type: VOTER last_known_addr { host: "127.28.154.188" port: 37317 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "41f9d60d712e4276b0da79549a2090e5" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 42507 } } peers { permanent_uuid: "169951c4d2524ade9f73f60b2d109e3a" member_type: NON_VOTER last_known_addr { host: "127.28.154.189" port: 33355 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "41f9d60d712e4276b0da79549a2090e5" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 42507 } } peers { permanent_uuid: "169951c4d2524ade9f73f60b2d109e3a" member_type: NON_VOTER last_known_addr { host: "127.28.154.189" port: 33355 } attrs { promote: true } } peers { permanent_uuid: "c6402a22cf5b441bbde21194d27f3d49" member_type: NON_VOTER last_known_addr { host: "127.28.154.188" port: 37317 } attrs { promote: true } }
I20260811 09:08:30.724921 9906 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41f9d60d712e4276b0da79549a2090e5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "41f9d60d712e4276b0da79549a2090e5" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "41f9d60d712e4276b0da79549a2090e5" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 42507 } } peers { permanent_uuid: "169951c4d2524ade9f73f60b2d109e3a" member_type: VOTER last_known_addr { host: "127.28.154.189" port: 33355 } attrs { promote: false } } peers { permanent_uuid: "c6402a22cf5b441bbde21194d27f3d49" member_type: NON_VOTER last_known_addr { host: "127.28.154.188" port: 37317 } attrs { promote: true } } }
I20260811 09:08:30.725111 9906 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41f9d60d712e4276b0da79549a2090e5 [sys.catalog]: This master's current role is: LEADER
I20260811 09:08:30.736918 9805 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c6402a22cf5b441bbde21194d27f3d49 [term 1 NON_PARTICIPANT]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 169951c4d2524ade9f73f60b2d109e3a (127.28.154.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "41f9d60d712e4276b0da79549a2090e5" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 42507 } } peers { permanent_uuid: "169951c4d2524ade9f73f60b2d109e3a" member_type: NON_VOTER last_known_addr { host: "127.28.154.189" port: 33355 } attrs { promote: true } } }
I20260811 09:08:30.743175 9917 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6402a22cf5b441bbde21194d27f3d49 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "41f9d60d712e4276b0da79549a2090e5" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "41f9d60d712e4276b0da79549a2090e5" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 42507 } } peers { permanent_uuid: "169951c4d2524ade9f73f60b2d109e3a" member_type: NON_VOTER last_known_addr { host: "127.28.154.189" port: 33355 } attrs { promote: true } } }
I20260811 09:08:30.743314 9867 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6402a22cf5b441bbde21194d27f3d49 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 41f9d60d712e4276b0da79549a2090e5. Latest consensus state: current_term: 1 leader_uuid: "41f9d60d712e4276b0da79549a2090e5" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "41f9d60d712e4276b0da79549a2090e5" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 42507 } } peers { permanent_uuid: "169951c4d2524ade9f73f60b2d109e3a" member_type: NON_VOTER last_known_addr { host: "127.28.154.189" port: 33355 } attrs { promote: true } } }
I20260811 09:08:30.743505 9917 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6402a22cf5b441bbde21194d27f3d49 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260811 09:08:30.743564 9867 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6402a22cf5b441bbde21194d27f3d49 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260811 09:08:30.743984 9920 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6402a22cf5b441bbde21194d27f3d49 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "41f9d60d712e4276b0da79549a2090e5" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "41f9d60d712e4276b0da79549a2090e5" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 42507 } } peers { permanent_uuid: "169951c4d2524ade9f73f60b2d109e3a" member_type: NON_VOTER last_known_addr { host: "127.28.154.189" port: 33355 } attrs { promote: true } } }
I20260811 09:08:30.744408 9920 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6402a22cf5b441bbde21194d27f3d49 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260811 09:08:30.745797 9566 raft_consensus.cc:2229] T 00000000000000000000000000000000 P c6402a22cf5b441bbde21194d27f3d49 [term 1 NON_PARTICIPANT]: Raft consensus shutting down.
I20260811 09:08:30.746301 9566 raft_consensus.cc:2258] T 00000000000000000000000000000000 P c6402a22cf5b441bbde21194d27f3d49 [term 1 NON_PARTICIPANT]: Raft consensus is shut down!
I20260811 09:08:30.746524 9566 tablet_replica.cc:330] T 00000000000000000000000000000000 P c6402a22cf5b441bbde21194d27f3d49: stopping tablet replica
I20260811 09:08:30.756256 9566 master.cc:552] Master@127.28.154.188:37317 shutdown complete.
I20260811 09:08:30.756392 9566 master_runner.cc:305] Clearing existing system tablet
I20260811 09:08:30.756804 9566 fs_manager.cc:359] Metadata directory not provided
I20260811 09:08:30.756909 9566 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:08:30.759912 9566 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260811 09:08:30.761920 9925 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:08:30.762401 9566 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:08:30.762928 9566 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260811 09:08:30.763098 9566 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/master-2/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/master-2/wal
uuid: "c6402a22cf5b441bbde21194d27f3d49"
format_stamp: "Formatted at 2026-08-11 09:08:30 on dist-test-slave-xt4k"
server_key: "18f654dfd0aef5f4b602be95882dd86d"
server_key_iv: "f9407f5eb909185f30692ad2822ce492"
server_key_version: "encryptionkey@0"
I20260811 09:08:30.768090 9566 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P c6402a22cf5b441bbde21194d27f3d49: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:08:30.777261 9566 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P c6402a22cf5b441bbde21194d27f3d49: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260811 09:08:30.777681 9566 log.cc:1196] T 00000000000000000000000000000000 P c6402a22cf5b441bbde21194d27f3d49: Deleting WAL directory at /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260811 09:08:30.778385 9566 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P c6402a22cf5b441bbde21194d27f3d49: Deleting consensus metadata
I20260811 09:08:30.778891 9566 master_runner.cc:315] Copying system tablet from 127.28.154.190:42507
I20260811 09:08:30.782979 9566 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P c6402a22cf5b441bbde21194d27f3d49: tablet copy: Beginning tablet copy session from remote peer at address 127.28.154.190:42507
I20260811 09:08:30.808735 9513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:30.784000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44746 (local address 127.28.154.190:42507)
0811 09:08:30.784242 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:30.784249 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:30.789067 (+ 4818us) server_negotiation.cc:407] Connection header received
0811 09:08:30.789365 (+ 298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:30.789371 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:30.789477 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:30.789581 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:30.790810 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:30.792141 (+ 1331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:30.794244 (+ 2103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:30.794972 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:30.795120 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:30.797991 (+ 2871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:30.798001 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:30.798006 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:08:30.798148 (+ 142us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:30.804305 (+ 6157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:30.805454 (+ 1149us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:30.805459 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:30.805463 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:30.805655 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:30.806203 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:30.806209 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:30.806213 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:30.806600 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:30.806749 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:30.808097 (+ 1348us) server_negotiation.cc:299] Negotiation successful
0811 09:08:30.808373 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116}
I20260811 09:08:30.809222 9937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:30.784000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:42507 (local address 127.0.0.1:44746)
0811 09:08:30.788890 (+ 4890us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:30.788924 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:30.789179 (+ 255us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:30.789643 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:30.789648 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:30.790233 (+ 585us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:08:30.790539 (+ 306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:30.790553 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:30.792380 (+ 1827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:30.792386 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:30.794072 (+ 1686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:30.794091 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:30.795156 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:30.795161 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:30.795336 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:30.796164 (+ 828us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:08:30.796188 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:08:30.797658 (+ 1470us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:08:30.804636 (+ 6978us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:08:30.804644 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:08:30.804649 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:08:30.805289 (+ 640us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:08:30.805797 (+ 508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:08:30.805803 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:08:30.805806 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:08:30.806053 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:08:30.806813 (+ 760us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:08:30.806818 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:08:30.807085 (+ 267us) client_negotiation.cc:769] Sending connection context
0811 09:08:30.807236 (+ 151us) client_negotiation.cc:240] Negotiation successful
0811 09:08:30.807595 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4675,"thread_start_us":4518,"threads_started":1}
I20260811 09:08:30.809834 8529 tablet_copy_service.cc:140] P 41f9d60d712e4276b0da79549a2090e5: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer c6402a22cf5b441bbde21194d27f3d49 ({username='oryx', principal='oryx/127.28.154.188@KRBTEST.COM'} at 127.0.0.1:44746)
I20260811 09:08:30.809978 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 8614
I20260811 09:08:30.810021 8529 tablet_copy_service.cc:161] P 41f9d60d712e4276b0da79549a2090e5: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer c6402a22cf5b441bbde21194d27f3d49 at {username='oryx', principal='oryx/127.28.154.188@KRBTEST.COM'} at 127.0.0.1:44746: session id = c6402a22cf5b441bbde21194d27f3d49-00000000000000000000000000000000
I20260811 09:08:30.813971 8529 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 41f9d60d712e4276b0da79549a2090e5: Tablet Copy: opened 0 blocks and 1 log segments
I20260811 09:08:30.818607 9566 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260811 09:08:30.826323 9566 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P c6402a22cf5b441bbde21194d27f3d49: tablet copy: Starting download of 0 data blocks...
I20260811 09:08:30.826666 9566 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P c6402a22cf5b441bbde21194d27f3d49: tablet copy: Starting download of 1 WAL segments...
I20260811 09:08:30.829483 9566 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P c6402a22cf5b441bbde21194d27f3d49: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260811 09:08:30.834226 8529 tablet_copy_service.cc:342] P 41f9d60d712e4276b0da79549a2090e5: Request end of tablet copy session c6402a22cf5b441bbde21194d27f3d49-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.28.154.188@KRBTEST.COM'} at 127.0.0.1:44746
I20260811 09:08:30.834381 8529 tablet_copy_service.cc:433] P 41f9d60d712e4276b0da79549a2090e5: ending tablet copy session c6402a22cf5b441bbde21194d27f3d49-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer c6402a22cf5b441bbde21194d27f3d49
I20260811 09:08:30.838976 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.154.129:34797
--local_ip_for_outbound_sockets=127.28.154.129
--tserver_master_addrs=127.28.154.190:42507
--webserver_port=37359
--webserver_interface=127.28.154.129
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/krb5kdc/oryx_127.28.154.129.keytab
--principal=oryx/127.28.154.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:33603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.28.154.190:42507,127.28.154.189:33355
--tserver_master_addrs=127.28.154.190:42507,127.28.154.189:33355,127.28.154.188:37317 with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 09:08:30.847043 9566 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:08:30.849869 9942 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:08:30.854316 9943 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:08:30.865607 9947 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:08:30.869843 9566 server_base.cc:1034] running on GCE node
I20260811 09:08:30.870321 9566 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:08:30.871057 9566 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:08:30.872267 9566 hybrid_clock.cc:648] HybridClock initialized: now 1786439310872224 us; error 31 us; skew 500 ppm
I20260811 09:08:30.872586 9566 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:08:30 dist-test-slave-xt4k krb5kdc[8413](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439310, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.154.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:08:30.874838 9566 init.cc:377] Logged in from keytab as oryx/127.28.154.188@KRBTEST.COM (short username oryx)
I20260811 09:08:30.877342 9566 webserver.cc:460] Webserver started at http://127.28.154.188:44565/ using document root <none> and password file <none>
I20260811 09:08:30.877684 9566 fs_manager.cc:359] Metadata directory not provided
I20260811 09:08:30.877903 9566 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:08:30.881189 9566 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260811 09:08:30.883124 9958 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:08:30.883535 9566 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20260811 09:08:30.883654 9566 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/master-2/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/master-2/wal
uuid: "c6402a22cf5b441bbde21194d27f3d49"
format_stamp: "Formatted at 2026-08-11 09:08:30 on dist-test-slave-xt4k"
server_key: "18f654dfd0aef5f4b602be95882dd86d"
server_key_iv: "f9407f5eb909185f30692ad2822ce492"
server_key_version: "encryptionkey@0"
I20260811 09:08:30.883836 9566 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:08:30.937018 9566 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:08:30.941318 9566 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:08:30.968531 9566 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.188:37317
I20260811 09:08:30.968587 10041 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.188:37317 every 8 connection(s)
I20260811 09:08:30.969693 9566 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/master-2/data/info.pb
I20260811 09:08:30.975055 10044 sys_catalog.cc:263] Verifying existing consensus state
I20260811 09:08:30.978838 10044 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c6402a22cf5b441bbde21194d27f3d49: Bootstrap starting.
I20260811 09:08:30.995635 10044 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c6402a22cf5b441bbde21194d27f3d49: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 09:08:30.996265 10044 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c6402a22cf5b441bbde21194d27f3d49: Bootstrap complete.
I20260811 09:08:30.997539 10044 raft_consensus.cc:348] T 00000000000000000000000000000000 P c6402a22cf5b441bbde21194d27f3d49 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "41f9d60d712e4276b0da79549a2090e5" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 42507 } } peers { permanent_uuid: "169951c4d2524ade9f73f60b2d109e3a" member_type: VOTER last_known_addr { host: "127.28.154.189" port: 33355 } attrs { promote: false } } peers { permanent_uuid: "c6402a22cf5b441bbde21194d27f3d49" member_type: NON_VOTER last_known_addr { host: "127.28.154.188" port: 37317 } attrs { promote: true } }
I20260811 09:08:30.997846 10044 raft_consensus.cc:729] T 00000000000000000000000000000000 P c6402a22cf5b441bbde21194d27f3d49 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c6402a22cf5b441bbde21194d27f3d49, State: Initialized, Role: LEARNER
I20260811 09:08:30.998116 10044 consensus_queue.cc:260] T 00000000000000000000000000000000 P c6402a22cf5b441bbde21194d27f3d49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "41f9d60d712e4276b0da79549a2090e5" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 42507 } } peers { permanent_uuid: "169951c4d2524ade9f73f60b2d109e3a" member_type: VOTER last_known_addr { host: "127.28.154.189" port: 33355 } attrs { promote: false } } peers { permanent_uuid: "c6402a22cf5b441bbde21194d27f3d49" member_type: NON_VOTER last_known_addr { host: "127.28.154.188" port: 37317 } attrs { promote: true } }
I20260811 09:08:30.999145 10048 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6402a22cf5b441bbde21194d27f3d49 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "41f9d60d712e4276b0da79549a2090e5" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 42507 } } peers { permanent_uuid: "169951c4d2524ade9f73f60b2d109e3a" member_type: VOTER last_known_addr { host: "127.28.154.189" port: 33355 } attrs { promote: false } } peers { permanent_uuid: "c6402a22cf5b441bbde21194d27f3d49" member_type: NON_VOTER last_known_addr { host: "127.28.154.188" port: 37317 } attrs { promote: true } } }
I20260811 09:08:30.999511 10044 sys_catalog.cc:564] T 00000000000000000000000000000000 P c6402a22cf5b441bbde21194d27f3d49 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:08:30.999699 10048 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6402a22cf5b441bbde21194d27f3d49 [sys.catalog]: This master's current role is: LEARNER
I20260811 09:08:31.011757 10060 catalog_manager.cc:1259] Loaded cluster ID: 039df53890a0413897b986c5c7822102
I20260811 09:08:31.011881 10060 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c6402a22cf5b441bbde21194d27f3d49: loading cluster ID for follower catalog manager: success
I20260811 09:08:31.015667 10060 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c6402a22cf5b441bbde21194d27f3d49: acquiring CA information for follower catalog manager: success
I20260811 09:08:31.019038 10060 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c6402a22cf5b441bbde21194d27f3d49: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 09:08:31.100530 9865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:31.089364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.188:37317 (local address 127.0.0.1:49410)
0811 09:08:31.089926 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:31.089955 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:31.090123 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:31.090924 (+ 801us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:31.090929 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:31.090970 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:08:31.091211 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:31.091224 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:31.093613 (+ 2389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:31.093619 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:31.096140 (+ 2521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:31.096173 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:31.099458 (+ 3285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:31.099467 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:31.099721 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:31.099766 (+ 45us) client_negotiation.cc:769] Sending connection context
0811 09:08:31.099884 (+ 118us) client_negotiation.cc:240] Negotiation successful
0811 09:08:31.100092 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":427}
I20260811 09:08:31.100522 10062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:31.089571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49410 (local address 127.28.154.188:37317)
0811 09:08:31.090328 (+ 757us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:31.090334 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:31.090364 (+ 30us) server_negotiation.cc:407] Connection header received
0811 09:08:31.090452 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:31.090458 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:31.090563 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:31.090904 (+ 341us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:08:31.091384 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:31.093402 (+ 2018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:31.096458 (+ 3056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:31.099079 (+ 2621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:31.099500 (+ 421us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:31.099775 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:31.099947 (+ 172us) server_negotiation.cc:299] Negotiation successful
0811 09:08:31.100104 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":585,"thread_start_us":686,"threads_started":1}
I20260811 09:08:31.101859 10003 raft_consensus.cc:1203] T 00000000000000000000000000000000 P c6402a22cf5b441bbde21194d27f3d49 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260811 09:08:31.102692 10048 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6402a22cf5b441bbde21194d27f3d49 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 41f9d60d712e4276b0da79549a2090e5. Latest consensus state: current_term: 1 leader_uuid: "41f9d60d712e4276b0da79549a2090e5" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "41f9d60d712e4276b0da79549a2090e5" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 42507 } } peers { permanent_uuid: "169951c4d2524ade9f73f60b2d109e3a" member_type: VOTER last_known_addr { host: "127.28.154.189" port: 33355 } attrs { promote: false } } peers { permanent_uuid: "c6402a22cf5b441bbde21194d27f3d49" member_type: NON_VOTER last_known_addr { host: "127.28.154.188" port: 37317 } attrs { promote: true } } }
I20260811 09:08:31.103011 10048 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6402a22cf5b441bbde21194d27f3d49 [sys.catalog]: This master's current role is: LEARNER
W20260811 09:08:31.243081 9941 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:08:31.243418 9941 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:08:31.243563 9941 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:08:31.251549 9941 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:08:31.251677 9941 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:08:31.251819 9941 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.154.129
I20260811 09:08:31.262655 9941 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:33603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/krb5kdc/oryx_127.28.154.129.keytab
--principal=oryx/127.28.154.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.154.129:34797
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.154.129
--webserver_port=37359
--tserver_master_addrs=127.28.154.190:42507,127.28.154.189:33355,127.28.154.188:37317
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.154.129
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:08:31.264102 9941 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:08:31.265633 9941 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:08:31.282342 10070 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:08:31.283099 10071 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:08:31.283259 10075 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:08:31.284662 9941 server_base.cc:1034] running on GCE node
I20260811 09:08:31.285513 9941 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:08:31.286581 9941 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:08:31.287837 9941 hybrid_clock.cc:648] HybridClock initialized: now 1786439311287760 us; error 47 us; skew 500 ppm
I20260811 09:08:31.288231 9941 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:08:31 dist-test-slave-xt4k krb5kdc[8413](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439311, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.154.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:08:31.291767 9941 init.cc:377] Logged in from keytab as oryx/127.28.154.129@KRBTEST.COM (short username oryx)
I20260811 09:08:31.293397 9941 webserver.cc:460] Webserver started at http://127.28.154.129:37359/ using document root <none> and password file <none>
I20260811 09:08:31.294142 9941 fs_manager.cc:359] Metadata directory not provided
I20260811 09:08:31.294279 9941 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:08:31.300282 9941 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 09:08:31.303529 10092 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:08:31.305184 9941 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 09:08:31.305351 9941 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/ts-0/wal
uuid: "2ed5f79097f84d8ea5faca70a459d1ca"
format_stamp: "Formatted at 2026-08-11 09:08:28 on dist-test-slave-xt4k"
server_key: "d14c96be2fd72fe57ec905c2353e10c3"
server_key_iv: "94def2a0bae551c8ba4a71d186c86e2d"
server_key_version: "encryptionkey@0"
I20260811 09:08:31.306468 9941 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:08:31.324662 9941 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:08:31.329262 9941 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:08:31.329519 9941 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:08:31.330581 9941 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:08:31.332033 9941 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:08:31.332121 9941 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:08:31.332207 9941 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:08:31.332267 9941 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:08:31.370998 9941 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.129:34797
I20260811 09:08:31.371064 10290 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.129:34797 every 8 connection(s)
I20260811 09:08:31.372656 9941 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/ts-0/data/info.pb
I20260811 09:08:31.380370 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 9941
I20260811 09:08:31.380739 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 8861
Aug 11 09:08:31 dist-test-slave-xt4k krb5kdc[8413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439311, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.154.129@KRBTEST.COM for oryx/127.28.154.188@KRBTEST.COM
Aug 11 09:08:31 dist-test-slave-xt4k krb5kdc[8413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439311, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.154.129@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM
Aug 11 09:08:31 dist-test-slave-xt4k krb5kdc[8413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439311, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.154.129@KRBTEST.COM for oryx/127.28.154.189@KRBTEST.COM
I20260811 09:08:31.405654 10303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:31.376498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.129:40427 (local address 127.28.154.190:42507)
0811 09:08:31.378481 (+ 1983us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:31.378487 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:31.383644 (+ 5157us) server_negotiation.cc:407] Connection header received
0811 09:08:31.383712 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:31.383717 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:31.383816 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:31.383923 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:31.385925 (+ 2002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:31.387120 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:31.390131 (+ 3011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:31.390832 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:31.390914 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:31.397394 (+ 6480us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:31.397403 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:31.397407 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:08:31.397456 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:31.401870 (+ 4414us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:31.403015 (+ 1145us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:31.403021 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:31.403025 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:31.403176 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:31.403648 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:31.403653 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:31.403657 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:31.404024 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:31.404156 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:31.404986 (+ 830us) server_negotiation.cc:299] Negotiation successful
0811 09:08:31.405216 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1850,"thread_start_us":1750,"threads_started":1}
I20260811 09:08:31.405763 10062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:31.376950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.129:43819 (local address 127.28.154.188:37317)
0811 09:08:31.377167 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:31.377174 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:31.379260 (+ 2086us) server_negotiation.cc:407] Connection header received
0811 09:08:31.382906 (+ 3646us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:31.382911 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:31.383007 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:31.383109 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:31.385926 (+ 2817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:31.387113 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:31.390133 (+ 3020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:31.390803 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:31.390913 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:31.397434 (+ 6521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:31.397442 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:31.397447 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:08:31.397493 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:31.401351 (+ 3858us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:31.402430 (+ 1079us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:31.402436 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:31.402441 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:31.402630 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:31.403280 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:31.403286 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:31.403289 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:31.403691 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:31.403832 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:31.405011 (+ 1179us) server_negotiation.cc:299] Negotiation successful
0811 09:08:31.405299 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
Aug 11 09:08:31 dist-test-slave-xt4k krb5kdc[8413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439306, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.28.154.130@KRBTEST.COM
I20260811 09:08:31.408512 10306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:31.376169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.188:37317 (local address 127.28.154.129:43819)
0811 09:08:31.379069 (+ 2900us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:31.379131 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:31.382267 (+ 3136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:31.383616 (+ 1349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:31.383629 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:31.384607 (+ 978us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:08:31.385696 (+ 1089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:31.385725 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:31.387319 (+ 1594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:31.387324 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:31.389954 (+ 2630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:31.389975 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:31.391321 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:31.391330 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:31.391693 (+ 363us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:31.392653 (+ 960us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:08:31.392712 (+ 59us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:08:31.397134 (+ 4422us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:08:31.401521 (+ 4387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:08:31.401532 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:08:31.401552 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:08:31.402251 (+ 699us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:08:31.402781 (+ 530us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:08:31.402785 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:08:31.402788 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:08:31.403128 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:08:31.403939 (+ 811us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:08:31.403948 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:08:31.404497 (+ 549us) client_negotiation.cc:769] Sending connection context
0811 09:08:31.406059 (+ 1562us) client_negotiation.cc:240] Negotiation successful
0811 09:08:31.407004 (+ 945us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2542,"mutex_wait_us":641,"spinlock_wait_cycles":22528,"thread_start_us":6800,"threads_started":1}
I20260811 09:08:31.408506 10307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:31.376169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:42507 (local address 127.28.154.129:40427)
0811 09:08:31.383386 (+ 7217us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:31.383403 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:31.383521 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:31.384169 (+ 648us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:31.384174 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:31.384983 (+ 809us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:08:31.385696 (+ 713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:31.385725 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:31.387319 (+ 1594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:31.387324 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:31.389955 (+ 2631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:31.389975 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:31.391321 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:31.391330 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:31.391691 (+ 361us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:31.392696 (+ 1005us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:08:31.392721 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:08:31.397058 (+ 4337us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:08:31.402041 (+ 4983us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:08:31.402045 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:08:31.402049 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:08:31.402867 (+ 818us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:08:31.403333 (+ 466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:08:31.403338 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:08:31.403341 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:08:31.403501 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:08:31.404241 (+ 740us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:08:31.404246 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:08:31.404491 (+ 245us) client_negotiation.cc:769] Sending connection context
0811 09:08:31.406004 (+ 1513us) client_negotiation.cc:240] Negotiation successful
0811 09:08:31.406855 (+ 851us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7053,"mutex_wait_us":323,"spinlock_wait_cycles":56960,"thread_start_us":19093,"threads_started":1}
I20260811 09:08:31.410362 10292 heartbeater.cc:347] Connected to a master server at 127.28.154.190:42507
I20260811 09:08:31.410686 10292 heartbeater.cc:464] Registering TS with master...
I20260811 09:08:31.411595 10292 heartbeater.cc:511] Master 127.28.154.190:42507 requested a full tablet report, sending...
I20260811 09:08:31.411770 10291 heartbeater.cc:347] Connected to a master server at 127.28.154.188:37317
I20260811 09:08:31.411897 10291 heartbeater.cc:464] Registering TS with master...
I20260811 09:08:31.412422 10291 heartbeater.cc:511] Master 127.28.154.188:37317 requested a full tablet report, sending...
I20260811 09:08:31.413790 8495 ts_manager.cc:194] Re-registered known tserver with Master: 2ed5f79097f84d8ea5faca70a459d1ca (127.28.154.129:34797)
I20260811 09:08:31.414369 9988 ts_manager.cc:194] Registered new tserver with Master: 2ed5f79097f84d8ea5faca70a459d1ca (127.28.154.129:34797)
I20260811 09:08:31.414644 10305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:31.376622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.129:45091 (local address 127.28.154.189:33355)
0811 09:08:31.392919 (+ 16297us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:31.392926 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:31.395854 (+ 2928us) server_negotiation.cc:407] Connection header received
0811 09:08:31.396046 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:31.396051 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:31.396174 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:31.396281 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:31.397729 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:31.399062 (+ 1333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:31.401130 (+ 2068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:31.401773 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:31.401872 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:31.406746 (+ 4874us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:31.406756 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:31.406761 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:08:31.406807 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:31.410739 (+ 3932us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:31.412160 (+ 1421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:31.412167 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:31.412170 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:31.412374 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:31.412905 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:31.412911 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:31.412914 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:31.413306 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:31.413447 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:31.414008 (+ 561us) server_negotiation.cc:299] Negotiation successful
0811 09:08:31.414242 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16019,"thread_start_us":16097,"threads_started":1}
I20260811 09:08:31.414906 8495 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.28.154.129@KRBTEST.COM'} at 127.28.154.129:40427
I20260811 09:08:31.415663 10302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:31.377544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.189:33355 (local address 127.28.154.129:45091)
0811 09:08:31.395739 (+ 18195us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:31.395759 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:31.395880 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:31.396540 (+ 660us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:31.396545 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:31.397223 (+ 678us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:08:31.397558 (+ 335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:31.397571 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:31.399300 (+ 1729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:31.399305 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:31.400942 (+ 1637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:31.400958 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:31.402178 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:31.402184 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:31.402410 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:31.402934 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:08:31.402957 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:08:31.406406 (+ 3449us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:08:31.411233 (+ 4827us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:08:31.411238 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:08:31.411241 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:08:31.412003 (+ 762us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:08:31.412538 (+ 535us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:08:31.412543 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:08:31.412547 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:08:31.412771 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:08:31.413537 (+ 766us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:08:31.413542 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:08:31.413705 (+ 163us) client_negotiation.cc:769] Sending connection context
0811 09:08:31.414897 (+ 1192us) client_negotiation.cc:240] Negotiation successful
0811 09:08:31.415222 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18070,"thread_start_us":1166,"threads_started":1}
I20260811 09:08:31.416663 10296 heartbeater.cc:347] Connected to a master server at 127.28.154.189:33355
I20260811 09:08:31.416803 10296 heartbeater.cc:464] Registering TS with master...
I20260811 09:08:31.417224 10296 heartbeater.cc:511] Master 127.28.154.189:33355 requested a full tablet report, sending...
I20260811 09:08:31.418200 10314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:31.381696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49044 (local address 127.28.154.130:35019)
0811 09:08:31.394049 (+ 12353us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:31.394060 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:31.394094 (+ 34us) server_negotiation.cc:407] Connection header received
0811 09:08:31.394528 (+ 434us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:31.394541 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:31.394882 (+ 341us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:31.395074 (+ 192us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:31.397550 (+ 2476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:31.399325 (+ 1775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:31.401557 (+ 2232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:31.402548 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:31.402673 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:31.409638 (+ 6965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:31.409658 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:31.409676 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:08:31.409734 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:31.414003 (+ 4269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:31.415096 (+ 1093us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:31.415105 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:31.415108 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:31.415311 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:31.416447 (+ 1136us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:31.416452 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:31.416455 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:31.416864 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:31.417170 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:31.417499 (+ 329us) server_negotiation.cc:299] Negotiation successful
0811 09:08:31.417766 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11594,"thread_start_us":11411,"threads_started":1}
I20260811 09:08:31.418816 9636 ts_manager.cc:194] Registered new tserver with Master: 2ed5f79097f84d8ea5faca70a459d1ca (127.28.154.129:34797)
I20260811 09:08:31.419464 9096 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:08:31.503146 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 8861
I20260811 09:08:31.511270 9906 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41f9d60d712e4276b0da79549a2090e5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "41f9d60d712e4276b0da79549a2090e5" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "41f9d60d712e4276b0da79549a2090e5" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 42507 } } peers { permanent_uuid: "169951c4d2524ade9f73f60b2d109e3a" member_type: VOTER last_known_addr { host: "127.28.154.189" port: 33355 } attrs { promote: false } } peers { permanent_uuid: "c6402a22cf5b441bbde21194d27f3d49" member_type: NON_VOTER last_known_addr { host: "127.28.154.188" port: 37317 } attrs { promote: true } } }
I20260811 09:08:31.511664 9906 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41f9d60d712e4276b0da79549a2090e5 [sys.catalog]: This master's current role is: LEADER
I20260811 09:08:31.516033 9906 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 41f9d60d712e4276b0da79549a2090e5: attempting to promote NON_VOTER c6402a22cf5b441bbde21194d27f3d49 to VOTER
I20260811 09:08:31.516669 9906 consensus_queue.cc:237] T 00000000000000000000000000000000 P 41f9d60d712e4276b0da79549a2090e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "41f9d60d712e4276b0da79549a2090e5" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 42507 } } peers { permanent_uuid: "169951c4d2524ade9f73f60b2d109e3a" member_type: VOTER last_known_addr { host: "127.28.154.189" port: 33355 } attrs { promote: false } } peers { permanent_uuid: "c6402a22cf5b441bbde21194d27f3d49" member_type: VOTER last_known_addr { host: "127.28.154.188" port: 37317 } attrs { promote: false } }
I20260811 09:08:31.518657 9656 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 169951c4d2524ade9f73f60b2d109e3a [term 1 FOLLOWER]: Refusing update from remote peer 41f9d60d712e4276b0da79549a2090e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260811 09:08:31.519490 9906 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 41f9d60d712e4276b0da79549a2090e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "169951c4d2524ade9f73f60b2d109e3a" member_type: VOTER last_known_addr { host: "127.28.154.189" port: 33355 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260811 09:08:31.519670 10003 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c6402a22cf5b441bbde21194d27f3d49 [term 1 LEARNER]: Refusing update from remote peer 41f9d60d712e4276b0da79549a2090e5: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260811 09:08:31.520433 10317 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 41f9d60d712e4276b0da79549a2090e5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c6402a22cf5b441bbde21194d27f3d49" member_type: VOTER last_known_addr { host: "127.28.154.188" port: 37317 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260811 09:08:31.522266 10317 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 41f9d60d712e4276b0da79549a2090e5 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, c6402a22cf5b441bbde21194d27f3d49 (127.28.154.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "41f9d60d712e4276b0da79549a2090e5" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 42507 } } peers { permanent_uuid: "169951c4d2524ade9f73f60b2d109e3a" member_type: VOTER last_known_addr { host: "127.28.154.189" port: 33355 } attrs { promote: false } } peers { permanent_uuid: "c6402a22cf5b441bbde21194d27f3d49" member_type: VOTER last_known_addr { host: "127.28.154.188" port: 37317 } attrs { promote: false } } }
I20260811 09:08:31.523321 9656 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 169951c4d2524ade9f73f60b2d109e3a [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, c6402a22cf5b441bbde21194d27f3d49 (127.28.154.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "41f9d60d712e4276b0da79549a2090e5" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 42507 } } peers { permanent_uuid: "169951c4d2524ade9f73f60b2d109e3a" member_type: VOTER last_known_addr { host: "127.28.154.189" port: 33355 } attrs { promote: false } } peers { permanent_uuid: "c6402a22cf5b441bbde21194d27f3d49" member_type: VOTER last_known_addr { host: "127.28.154.188" port: 37317 } attrs { promote: false } } }
I20260811 09:08:31.532094 10321 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41f9d60d712e4276b0da79549a2090e5 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "41f9d60d712e4276b0da79549a2090e5" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "41f9d60d712e4276b0da79549a2090e5" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 42507 } } peers { permanent_uuid: "169951c4d2524ade9f73f60b2d109e3a" member_type: VOTER last_known_addr { host: "127.28.154.189" port: 33355 } attrs { promote: false } } peers { permanent_uuid: "c6402a22cf5b441bbde21194d27f3d49" member_type: VOTER last_known_addr { host: "127.28.154.188" port: 37317 } attrs { promote: false } } }
I20260811 09:08:31.532547 10321 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41f9d60d712e4276b0da79549a2090e5 [sys.catalog]: This master's current role is: LEADER
I20260811 09:08:31.532677 9906 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41f9d60d712e4276b0da79549a2090e5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "41f9d60d712e4276b0da79549a2090e5" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "41f9d60d712e4276b0da79549a2090e5" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 42507 } } peers { permanent_uuid: "169951c4d2524ade9f73f60b2d109e3a" member_type: VOTER last_known_addr { host: "127.28.154.189" port: 33355 } attrs { promote: false } } peers { permanent_uuid: "c6402a22cf5b441bbde21194d27f3d49" member_type: VOTER last_known_addr { host: "127.28.154.188" port: 37317 } attrs { promote: false } } }
I20260811 09:08:31.532986 9906 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41f9d60d712e4276b0da79549a2090e5 [sys.catalog]: This master's current role is: LEADER
I20260811 09:08:31.533413 9906 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41f9d60d712e4276b0da79549a2090e5 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "41f9d60d712e4276b0da79549a2090e5" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "41f9d60d712e4276b0da79549a2090e5" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 42507 } } peers { permanent_uuid: "169951c4d2524ade9f73f60b2d109e3a" member_type: VOTER last_known_addr { host: "127.28.154.189" port: 33355 } attrs { promote: false } } peers { permanent_uuid: "c6402a22cf5b441bbde21194d27f3d49" member_type: VOTER last_known_addr { host: "127.28.154.188" port: 37317 } attrs { promote: false } } }
I20260811 09:08:31.534015 9906 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41f9d60d712e4276b0da79549a2090e5 [sys.catalog]: This master's current role is: LEADER
I20260811 09:08:31.534214 10003 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c6402a22cf5b441bbde21194d27f3d49 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, c6402a22cf5b441bbde21194d27f3d49 (127.28.154.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "41f9d60d712e4276b0da79549a2090e5" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 42507 } } peers { permanent_uuid: "169951c4d2524ade9f73f60b2d109e3a" member_type: VOTER last_known_addr { host: "127.28.154.189" port: 33355 } attrs { promote: false } } peers { permanent_uuid: "c6402a22cf5b441bbde21194d27f3d49" member_type: VOTER last_known_addr { host: "127.28.154.188" port: 37317 } attrs { promote: false } } }
I20260811 09:08:31.536405 10330 sys_catalog.cc:455] T 00000000000000000000000000000000 P 169951c4d2524ade9f73f60b2d109e3a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "41f9d60d712e4276b0da79549a2090e5" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "41f9d60d712e4276b0da79549a2090e5" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 42507 } } peers { permanent_uuid: "169951c4d2524ade9f73f60b2d109e3a" member_type: VOTER last_known_addr { host: "127.28.154.189" port: 33355 } attrs { promote: false } } peers { permanent_uuid: "c6402a22cf5b441bbde21194d27f3d49" member_type: VOTER last_known_addr { host: "127.28.154.188" port: 37317 } attrs { promote: false } } }
I20260811 09:08:31.536916 10330 sys_catalog.cc:458] T 00000000000000000000000000000000 P 169951c4d2524ade9f73f60b2d109e3a [sys.catalog]: This master's current role is: FOLLOWER
I20260811 09:08:31.537293 10048 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6402a22cf5b441bbde21194d27f3d49 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "41f9d60d712e4276b0da79549a2090e5" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "41f9d60d712e4276b0da79549a2090e5" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 42507 } } peers { permanent_uuid: "169951c4d2524ade9f73f60b2d109e3a" member_type: VOTER last_known_addr { host: "127.28.154.189" port: 33355 } attrs { promote: false } } peers { permanent_uuid: "c6402a22cf5b441bbde21194d27f3d49" member_type: VOTER last_known_addr { host: "127.28.154.188" port: 37317 } attrs { promote: false } } }
I20260811 09:08:31.537565 10048 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6402a22cf5b441bbde21194d27f3d49 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 09:08:31.538177 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.154.130:35019
--local_ip_for_outbound_sockets=127.28.154.130
--tserver_master_addrs=127.28.154.190:42507
--webserver_port=35155
--webserver_interface=127.28.154.130
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/krb5kdc/oryx_127.28.154.130.keytab
--principal=oryx/127.28.154.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:33603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.28.154.190:42507,127.28.154.189:33355
--tserver_master_addrs=127.28.154.190:42507,127.28.154.189:33355,127.28.154.188:37317 with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:08:31.879254 10335 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:08:31.879590 10335 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:08:31.879705 10335 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:08:31.887293 10335 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:08:31.887419 10335 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:08:31.887583 10335 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.154.130
I20260811 09:08:31.897909 10335 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:33603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/krb5kdc/oryx_127.28.154.130.keytab
--principal=oryx/127.28.154.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.154.130:35019
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.154.130
--webserver_port=35155
--tserver_master_addrs=127.28.154.190:42507,127.28.154.189:33355,127.28.154.188:37317
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.154.130
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:08:31.899272 10335 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:08:31.900772 10335 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:08:31.914191 10349 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:08:31.914243 10352 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:08:31.914283 10345 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:08:31.915151 10335 server_base.cc:1034] running on GCE node
I20260811 09:08:31.915783 10335 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:08:31.916761 10335 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:08:31.917961 10335 hybrid_clock.cc:648] HybridClock initialized: now 1786439311917913 us; error 42 us; skew 500 ppm
I20260811 09:08:31.918284 10335 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:08:31 dist-test-slave-xt4k krb5kdc[8413](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439311, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.154.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:08:31.921514 10335 init.cc:377] Logged in from keytab as oryx/127.28.154.130@KRBTEST.COM (short username oryx)
I20260811 09:08:31.922991 10335 webserver.cc:460] Webserver started at http://127.28.154.130:35155/ using document root <none> and password file <none>
I20260811 09:08:31.923666 10335 fs_manager.cc:359] Metadata directory not provided
I20260811 09:08:31.923795 10335 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:08:31.929168 10335 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260811 09:08:31.931977 10366 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:08:31.933415 10335 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 09:08:31.933575 10335 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/ts-1/wal
uuid: "5675cf4144864bd9aa773a78d2e2ed50"
format_stamp: "Formatted at 2026-08-11 09:08:28 on dist-test-slave-xt4k"
server_key: "a513d67546da7d095f1fae24fda2e4ec"
server_key_iv: "5153fafe851d0e0544e789ffd70c2b8e"
server_key_version: "encryptionkey@0"
I20260811 09:08:31.934546 10335 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:08:31.949013 10335 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:08:31.953333 10335 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:08:31.953635 10335 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:08:31.954660 10335 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:08:31.956094 10335 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:08:31.956197 10335 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:08:31.956310 10335 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:08:31.956400 10335 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260811 09:08:31.989523 10335 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.130:35019
I20260811 09:08:31.989610 10567 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.130:35019 every 8 connection(s)
I20260811 09:08:31.990952 10335 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/ts-1/data/info.pb
I20260811 09:08:31.992861 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 10335
I20260811 09:08:31.993175 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 9116
Aug 11 09:08:32 dist-test-slave-xt4k krb5kdc[8413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439306, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.28.154.131@KRBTEST.COM
Aug 11 09:08:32 dist-test-slave-xt4k krb5kdc[8413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439311, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.154.130@KRBTEST.COM for oryx/127.28.154.189@KRBTEST.COM
Aug 11 09:08:32 dist-test-slave-xt4k krb5kdc[8413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439311, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.154.130@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM
Aug 11 09:08:32 dist-test-slave-xt4k krb5kdc[8413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439311, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.154.130@KRBTEST.COM for oryx/127.28.154.188@KRBTEST.COM
I20260811 09:08:32.042021 10578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:31.994181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48896 (local address 127.28.154.131:39329)
0811 09:08:32.018265 (+ 24084us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:32.018276 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:32.018940 (+ 664us) server_negotiation.cc:407] Connection header received
0811 09:08:32.019138 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:32.019147 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:32.019451 (+ 304us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:32.019619 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:32.022141 (+ 2522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:32.023349 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:32.025295 (+ 1946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:32.026218 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:32.026327 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:32.033694 (+ 7367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:32.033711 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:32.033728 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:08:32.033772 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:32.038044 (+ 4272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:32.039428 (+ 1384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:32.039440 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:32.039444 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:32.039640 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:32.040200 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:32.040204 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:32.040207 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:32.040678 (+ 471us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:32.040994 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:32.041331 (+ 337us) server_negotiation.cc:299] Negotiation successful
0811 09:08:32.041620 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23876,"thread_start_us":24910,"threads_started":1}
I20260811 09:08:32.043534 9358 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:08:32.044275 10576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:31.994082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.130:60465 (local address 127.28.154.189:33355)
0811 09:08:32.015687 (+ 21605us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:32.015694 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:32.020206 (+ 4512us) server_negotiation.cc:407] Connection header received
0811 09:08:32.020392 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:32.020397 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:32.020488 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:32.020609 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:32.024193 (+ 3584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:32.025330 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:32.027309 (+ 1979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:32.028062 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:32.028162 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:32.034758 (+ 6596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:32.034766 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:32.034771 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:08:32.034843 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:32.038693 (+ 3850us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:32.041899 (+ 3206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:32.041906 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:32.041909 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:32.042075 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:32.042535 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:32.042540 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:32.042543 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:32.042918 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:32.043066 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:32.043590 (+ 524us) server_negotiation.cc:299] Negotiation successful
0811 09:08:32.043920 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21458,"thread_start_us":21601,"threads_started":1}
I20260811 09:08:32.045271 10588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:32.000576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.130:58329 (local address 127.28.154.188:37317)
0811 09:08:32.020876 (+ 20300us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:32.020883 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:32.020915 (+ 32us) server_negotiation.cc:407] Connection header received
0811 09:08:32.020982 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:32.020987 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:32.021066 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:32.022015 (+ 949us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:32.023959 (+ 1944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:32.025141 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:32.029986 (+ 4845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:32.030785 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:32.030919 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:32.035991 (+ 5072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:32.036001 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:32.036007 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:08:32.036055 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:32.040283 (+ 4228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:32.041437 (+ 1154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:32.041443 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:32.041446 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:32.041609 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:32.042162 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:32.042168 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:32.042171 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:32.042587 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:32.042714 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:32.044662 (+ 1948us) server_negotiation.cc:299] Negotiation successful
0811 09:08:32.044899 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20122,"thread_start_us":19961,"threads_started":1}
I20260811 09:08:32.046375 10581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:31.999932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.188:37317 (local address 127.28.154.130:58329)
0811 09:08:32.016712 (+ 16780us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:32.016821 (+ 109us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:32.019666 (+ 2845us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:32.021651 (+ 1985us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:32.021656 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:32.022596 (+ 940us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:08:32.023749 (+ 1153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:32.023778 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:32.026588 (+ 2810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:32.026594 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:32.029781 (+ 3187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:32.029798 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:32.031267 (+ 1469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:32.031275 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:32.031492 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:32.032017 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:08:32.032039 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:08:32.035604 (+ 3565us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:08:32.040794 (+ 5190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:08:32.040800 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:08:32.040803 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:08:32.041306 (+ 503us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:08:32.041755 (+ 449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:08:32.041760 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:08:32.041763 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:08:32.042007 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:08:32.042723 (+ 716us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:08:32.042731 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:08:32.043142 (+ 411us) client_negotiation.cc:769] Sending connection context
0811 09:08:32.043457 (+ 315us) client_negotiation.cc:240] Negotiation successful
0811 09:08:32.044118 (+ 661us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16465,"mutex_wait_us":1255,"thread_start_us":21607,"threads_started":1}
I20260811 09:08:32.047173 10582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:32.000584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:42507 (local address 127.28.154.130:57459)
0811 09:08:32.022960 (+ 22376us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:32.022990 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:32.023136 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:32.024180 (+ 1044us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:32.024184 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:32.024852 (+ 668us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:08:32.025236 (+ 384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:32.025250 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:32.026594 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:32.026600 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:32.028562 (+ 1962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:32.028575 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:32.029671 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:32.029677 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:32.029872 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:32.030465 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:08:32.030489 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:08:32.035674 (+ 5185us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:08:32.040255 (+ 4581us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:08:32.040265 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:08:32.040285 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:08:32.041159 (+ 874us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:08:32.044519 (+ 3360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:08:32.044524 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:08:32.044528 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:08:32.044740 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:08:32.046146 (+ 1406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:08:32.046152 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:08:32.046313 (+ 161us) client_negotiation.cc:769] Sending connection context
0811 09:08:32.046488 (+ 175us) client_negotiation.cc:240] Negotiation successful
0811 09:08:32.046764 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21418,"mutex_wait_us":64,"thread_start_us":16161,"threads_started":1}
I20260811 09:08:32.047325 10586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:32.000597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.130:57459 (local address 127.28.154.190:42507)
0811 09:08:32.015170 (+ 14573us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:32.015177 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:32.023108 (+ 7931us) server_negotiation.cc:407] Connection header received
0811 09:08:32.023243 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:32.023248 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:32.023344 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:32.023459 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:32.025392 (+ 1933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:32.026435 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:32.028700 (+ 2265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:32.029456 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:32.029594 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:32.035891 (+ 6297us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:32.035899 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:32.035910 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:08:32.035958 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:32.039792 (+ 3834us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:32.043231 (+ 3439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:32.043236 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:32.043239 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:32.043424 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:32.044885 (+ 1461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:32.044891 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:32.044895 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:32.045301 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:32.045461 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:32.046634 (+ 1173us) server_negotiation.cc:299] Negotiation successful
0811 09:08:32.046901 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14409,"thread_start_us":14608,"threads_started":1}
I20260811 09:08:32.047591 10589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:31.994811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.189:33355 (local address 127.28.154.130:60465)
0811 09:08:32.016712 (+ 21901us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:32.020091 (+ 3379us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:32.020228 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:32.021157 (+ 929us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:32.021172 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:32.023657 (+ 2485us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:08:32.024036 (+ 379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:32.024046 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:32.025467 (+ 1421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:32.025473 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:32.027122 (+ 1649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:32.027142 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:32.028599 (+ 1457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:32.028607 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:32.028889 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:32.029552 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:08:32.029585 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:08:32.034435 (+ 4850us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:08:32.040499 (+ 6064us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:08:32.040504 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:08:32.040507 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:08:32.041754 (+ 1247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:08:32.042220 (+ 466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:08:32.042225 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:08:32.042228 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:08:32.042395 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:08:32.043073 (+ 678us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:08:32.043077 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:08:32.043236 (+ 159us) client_negotiation.cc:769] Sending connection context
0811 09:08:32.044853 (+ 1617us) client_negotiation.cc:240] Negotiation successful
0811 09:08:32.045143 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21293,"mutex_wait_us":3,"thread_start_us":21145,"threads_started":1}
I20260811 09:08:32.048146 10568 heartbeater.cc:347] Connected to a master server at 127.28.154.188:37317
I20260811 09:08:32.048180 10570 heartbeater.cc:347] Connected to a master server at 127.28.154.190:42507
I20260811 09:08:32.048555 10570 heartbeater.cc:464] Registering TS with master...
I20260811 09:08:32.048789 10572 heartbeater.cc:347] Connected to a master server at 127.28.154.189:33355
I20260811 09:08:32.048923 10572 heartbeater.cc:464] Registering TS with master...
I20260811 09:08:32.049620 10572 heartbeater.cc:511] Master 127.28.154.189:33355 requested a full tablet report, sending...
I20260811 09:08:32.049623 10570 heartbeater.cc:511] Master 127.28.154.190:42507 requested a full tablet report, sending...
I20260811 09:08:32.050761 10568 heartbeater.cc:464] Registering TS with master...
I20260811 09:08:32.051206 10568 heartbeater.cc:511] Master 127.28.154.188:37317 requested a full tablet report, sending...
I20260811 09:08:32.051322 8495 ts_manager.cc:194] Re-registered known tserver with Master: 5675cf4144864bd9aa773a78d2e2ed50 (127.28.154.130:35019)
I20260811 09:08:32.051508 9636 ts_manager.cc:194] Registered new tserver with Master: 5675cf4144864bd9aa773a78d2e2ed50 (127.28.154.130:35019)
I20260811 09:08:32.052246 9988 ts_manager.cc:194] Registered new tserver with Master: 5675cf4144864bd9aa773a78d2e2ed50 (127.28.154.130:35019)
I20260811 09:08:32.052559 8495 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.28.154.130@KRBTEST.COM'} at 127.28.154.130:57459
I20260811 09:08:32.132862 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 9116
I20260811 09:08:32.155439 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.154.131:39329
--local_ip_for_outbound_sockets=127.28.154.131
--tserver_master_addrs=127.28.154.190:42507
--webserver_port=44757
--webserver_interface=127.28.154.131
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/krb5kdc/oryx_127.28.154.131.keytab
--principal=oryx/127.28.154.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:33603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.28.154.190:42507,127.28.154.189:33355
--tserver_master_addrs=127.28.154.190:42507,127.28.154.189:33355,127.28.154.188:37317 with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 09:08:32.418083 10292 heartbeater.cc:503] Master 127.28.154.190:42507 was elected leader, sending a full tablet report...
W20260811 09:08:32.505371 10597 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:08:32.505692 10597 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:08:32.505812 10597 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:08:32.513494 10597 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:08:32.513610 10597 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:08:32.513736 10597 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.154.131
I20260811 09:08:32.524040 10597 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:33603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/krb5kdc/oryx_127.28.154.131.keytab
--principal=oryx/127.28.154.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.154.131:39329
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.154.131
--webserver_port=44757
--tserver_master_addrs=127.28.154.190:42507,127.28.154.189:33355,127.28.154.188:37317
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.154.131
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:08:32.525425 10597 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:08:32.526875 10597 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:08:32.540899 10603 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:08:32.540982 10609 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:08:32.540869 10605 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:08:32.542126 10597 server_base.cc:1034] running on GCE node
I20260811 09:08:32.542910 10597 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:08:32.543922 10597 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:08:32.545147 10597 hybrid_clock.cc:648] HybridClock initialized: now 1786439312545075 us; error 63 us; skew 500 ppm
I20260811 09:08:32.545485 10597 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:08:32 dist-test-slave-xt4k krb5kdc[8413](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439312, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.154.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:08:32.549304 10597 init.cc:377] Logged in from keytab as oryx/127.28.154.131@KRBTEST.COM (short username oryx)
I20260811 09:08:32.550956 10597 webserver.cc:460] Webserver started at http://127.28.154.131:44757/ using document root <none> and password file <none>
I20260811 09:08:32.551668 10597 fs_manager.cc:359] Metadata directory not provided
I20260811 09:08:32.551812 10597 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:08:32.557853 10597 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 09:08:32.560999 10621 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:08:32.562541 10597 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 09:08:32.562876 10597 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/ts-2/wal
uuid: "c62d163d6d3a435b821da93a68c125f9"
format_stamp: "Formatted at 2026-08-11 09:08:29 on dist-test-slave-xt4k"
server_key: "bfa6a7b4222541dc7f9e54f6487a457a"
server_key_iv: "5b7d63cc7db4beabbbd82b558b4eda63"
server_key_version: "encryptionkey@0"
I20260811 09:08:32.563978 10597 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:08:32.580989 10597 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:08:32.585466 10597 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:08:32.585742 10597 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:08:32.586887 10597 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:08:32.588672 10597 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:08:32.588790 10597 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:08:32.588897 10597 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:08:32.588973 10597 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:08:32.620968 10597 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.131:39329
I20260811 09:08:32.621007 10830 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.131:39329 every 8 connection(s)
I20260811 09:08:32.622689 10597 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/ts-2/data/info.pb
I20260811 09:08:32.633100 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 10597
Aug 11 09:08:32 dist-test-slave-xt4k krb5kdc[8413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439312, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.154.131@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM
Aug 11 09:08:32 dist-test-slave-xt4k krb5kdc[8413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439312, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.154.131@KRBTEST.COM for oryx/127.28.154.189@KRBTEST.COM
Aug 11 09:08:32 dist-test-slave-xt4k krb5kdc[8413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439312, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.154.131@KRBTEST.COM for oryx/127.28.154.188@KRBTEST.COM
I20260811 09:08:32.691190 10837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:32.625761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.131:51903 (local address 127.28.154.190:42507)
0811 09:08:32.641917 (+ 16156us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:32.641924 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:32.660187 (+ 18263us) server_negotiation.cc:407] Connection header received
0811 09:08:32.667821 (+ 7634us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:32.667828 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:32.667925 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:32.668040 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:32.670335 (+ 2295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:32.671476 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:32.674359 (+ 2883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:32.675009 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:32.675137 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:32.680234 (+ 5097us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:32.680242 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:32.680246 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:08:32.680292 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:32.686057 (+ 5765us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:32.687295 (+ 1238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:32.687302 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:32.687306 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:32.687475 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:32.688113 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:32.688119 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:32.688121 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:32.688543 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:32.688706 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:32.690409 (+ 1703us) server_negotiation.cc:299] Negotiation successful
0811 09:08:32.690793 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15995,"thread_start_us":15866,"threads_started":1}
I20260811 09:08:32.696702 10847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:32.626397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:42507 (local address 127.28.154.131:51903)
0811 09:08:32.659904 (+ 33507us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:32.659988 (+ 84us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:32.666577 (+ 6589us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:32.668072 (+ 1495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:32.668076 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:32.669281 (+ 1205us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:08:32.670150 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:32.670174 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:32.672065 (+ 1891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:32.672069 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:32.674223 (+ 2154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:32.674240 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:32.675230 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:32.675237 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:32.675581 (+ 344us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:32.676493 (+ 912us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:08:32.676529 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:08:32.679946 (+ 3417us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:08:32.686251 (+ 6305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:08:32.686269 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:08:32.686296 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:08:32.687129 (+ 833us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:08:32.687621 (+ 492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:08:32.687625 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:08:32.687628 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:08:32.687995 (+ 367us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:08:32.689268 (+ 1273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:08:32.689279 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:08:32.689881 (+ 602us) client_negotiation.cc:769] Sending connection context
0811 09:08:32.690242 (+ 361us) client_negotiation.cc:240] Negotiation successful
0811 09:08:32.690948 (+ 706us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33020,"mutex_wait_us":3,"thread_start_us":37650,"threads_started":1}
I20260811 09:08:32.698686 10840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:32.626591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.131:39363 (local address 127.28.154.189:33355)
0811 09:08:32.668162 (+ 41571us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:32.668168 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:32.668185 (+ 17us) server_negotiation.cc:407] Connection header received
0811 09:08:32.668246 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:32.668250 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:32.668326 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:32.668456 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:32.670392 (+ 1936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:32.671432 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:32.679637 (+ 8205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:32.680516 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:32.680637 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:32.688041 (+ 7404us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:32.688049 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:32.688053 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:08:32.688107 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:32.692295 (+ 4188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:32.694598 (+ 2303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:32.694606 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:32.694610 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:32.694831 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:32.695718 (+ 887us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:32.695724 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:32.695728 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:32.696523 (+ 795us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:32.696677 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:32.697940 (+ 1263us) server_negotiation.cc:299] Negotiation successful
0811 09:08:32.698204 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41439,"thread_start_us":41287,"threads_started":1}
I20260811 09:08:32.699034 10833 heartbeater.cc:347] Connected to a master server at 127.28.154.190:42507
I20260811 09:08:32.699613 10833 heartbeater.cc:464] Registering TS with master...
I20260811 09:08:32.701020 10833 heartbeater.cc:511] Master 127.28.154.190:42507 requested a full tablet report, sending...
I20260811 09:08:32.703058 8495 ts_manager.cc:194] Re-registered known tserver with Master: c62d163d6d3a435b821da93a68c125f9 (127.28.154.131:39329)
I20260811 09:08:32.703140 10842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:32.626397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.189:33355 (local address 127.28.154.131:39363)
0811 09:08:32.664439 (+ 38042us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:32.664461 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:32.666925 (+ 2464us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:32.668484 (+ 1559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:32.668489 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:32.669398 (+ 909us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:08:32.670183 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:32.670198 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:32.671683 (+ 1485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:32.671689 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:32.678726 (+ 7037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:32.678748 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:32.681256 (+ 2508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:32.681264 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:32.681465 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:32.682667 (+ 1202us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:08:32.682692 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:08:32.687482 (+ 4790us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:08:32.693061 (+ 5579us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:08:32.693069 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:08:32.693074 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:08:32.693688 (+ 614us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:08:32.695029 (+ 1341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:08:32.695034 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:08:32.695038 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:08:32.695242 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:08:32.696794 (+ 1552us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:08:32.696800 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:08:32.697088 (+ 288us) client_negotiation.cc:769] Sending connection context
0811 09:08:32.699107 (+ 2019us) client_negotiation.cc:240] Negotiation successful
0811 09:08:32.699479 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37848,"mutex_wait_us":385,"thread_start_us":34600,"threads_started":1}
I20260811 09:08:32.704253 8495 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.28.154.131@KRBTEST.COM'} at 127.28.154.131:51903
I20260811 09:08:32.704438 10834 heartbeater.cc:347] Connected to a master server at 127.28.154.189:33355
I20260811 09:08:32.704597 10834 heartbeater.cc:464] Registering TS with master...
I20260811 09:08:32.705148 10834 heartbeater.cc:511] Master 127.28.154.189:33355 requested a full tablet report, sending...
I20260811 09:08:32.705191 10838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:32.626563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.131:40081 (local address 127.28.154.188:37317)
0811 09:08:32.667351 (+ 40788us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:32.667355 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:32.667371 (+ 16us) server_negotiation.cc:407] Connection header received
0811 09:08:32.667425 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:32.667429 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:32.667489 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:32.667599 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:32.670813 (+ 3214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:32.671837 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:32.680447 (+ 8610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:32.681152 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:32.681269 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:32.688274 (+ 7005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:32.688283 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:32.688288 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:08:32.688364 (+ 76us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:32.694638 (+ 6274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:32.701705 (+ 7067us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:32.701713 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:32.701718 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:32.701949 (+ 231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:32.702474 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:32.702480 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:32.702484 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:32.702953 (+ 469us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:32.703124 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:32.704532 (+ 1408us) server_negotiation.cc:299] Negotiation successful
0811 09:08:32.704790 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40672,"thread_start_us":40576,"threads_started":1}
I20260811 09:08:32.706617 9636 ts_manager.cc:194] Registered new tserver with Master: c62d163d6d3a435b821da93a68c125f9 (127.28.154.131:39329)
I20260811 09:08:32.707113 10845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:32.626651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.188:37317 (local address 127.28.154.131:40081)
0811 09:08:32.663978 (+ 37327us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:32.664007 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:32.666588 (+ 2581us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:32.667910 (+ 1322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:32.667923 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:32.670329 (+ 2406us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:08:32.670610 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:32.670621 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:32.674446 (+ 3825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:32.674451 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:32.680224 (+ 5773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:32.680235 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:32.681675 (+ 1440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:32.681682 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:32.681884 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:32.682589 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:08:32.682616 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:08:32.687464 (+ 4848us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:08:32.697929 (+ 10465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:08:32.697935 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:08:32.697939 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:08:32.701500 (+ 3561us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:08:32.702133 (+ 633us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:08:32.702139 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:08:32.702144 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:08:32.702321 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:08:32.703967 (+ 1646us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:08:32.703984 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:08:32.704178 (+ 194us) client_negotiation.cc:769] Sending connection context
0811 09:08:32.705415 (+ 1237us) client_negotiation.cc:240] Negotiation successful
0811 09:08:32.705735 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37150,"thread_start_us":36950,"threads_started":1}
I20260811 09:08:32.709288 10832 heartbeater.cc:347] Connected to a master server at 127.28.154.188:37317
I20260811 09:08:32.709827 10832 heartbeater.cc:464] Registering TS with master...
I20260811 09:08:32.710326 10832 heartbeater.cc:511] Master 127.28.154.188:37317 requested a full tablet report, sending...
I20260811 09:08:32.712363 9988 ts_manager.cc:194] Registered new tserver with Master: c62d163d6d3a435b821da93a68c125f9 (127.28.154.131:39329)
I20260811 09:08:32.725961 10858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:32.672156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44752 (local address 127.28.154.190:42507)
0811 09:08:32.672688 (+ 532us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:32.672694 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:32.704091 (+ 31397us) server_negotiation.cc:407] Connection header received
0811 09:08:32.704157 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:32.704163 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:32.704246 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:32.705501 (+ 1255us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:32.710429 (+ 4928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:32.711850 (+ 1421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:32.714825 (+ 2975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:32.715493 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:32.715625 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:32.719179 (+ 3554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:32.719185 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:32.719188 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:08:32.719226 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:32.722556 (+ 3330us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:32.723465 (+ 909us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:32.723471 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:32.723474 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:32.723656 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:32.724192 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:32.724197 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:32.724200 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:32.724617 (+ 417us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:32.724901 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:32.725263 (+ 362us) server_negotiation.cc:299] Negotiation successful
0811 09:08:32.725446 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":411,"thread_start_us":278,"threads_started":1}
I20260811 09:08:32.727540 10840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:32.673204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36126 (local address 127.28.154.189:33355)
0811 09:08:32.698963 (+ 25759us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:32.698969 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:32.710464 (+ 11495us) server_negotiation.cc:407] Connection header received
0811 09:08:32.710648 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:32.710654 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:32.710772 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:32.710979 (+ 207us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:32.713644 (+ 2665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:32.714634 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:32.716398 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:32.717074 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:32.717209 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:32.720753 (+ 3544us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:32.720760 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:32.720765 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:08:32.720805 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:32.723838 (+ 3033us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:32.724876 (+ 1038us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:32.724882 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:32.724885 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:32.725072 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:32.725527 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:32.725533 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:32.725536 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:32.726002 (+ 466us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:32.726296 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:32.726927 (+ 631us) server_negotiation.cc:299] Negotiation successful
0811 09:08:32.727173 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25644,"thread_start_us":36488,"threads_started":1}
I20260811 09:08:32.728972 10864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:32.673797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49418 (local address 127.28.154.188:37317)
0811 09:08:32.703030 (+ 29233us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:32.703037 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:32.707848 (+ 4811us) server_negotiation.cc:407] Connection header received
0811 09:08:32.708000 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:32.708003 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:32.708066 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:32.708169 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:32.713041 (+ 4872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:32.714061 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:32.715877 (+ 1816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:32.716488 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:32.716609 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:32.720461 (+ 3852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:32.720469 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:32.720474 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:08:32.720514 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:32.723718 (+ 3204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:32.724829 (+ 1111us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:32.724835 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:32.724838 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:32.725062 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:32.726503 (+ 1441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:32.726509 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:32.726512 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:32.726885 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:32.727135 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:32.728311 (+ 1176us) server_negotiation.cc:299] Negotiation successful
0811 09:08:32.728572 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29089,"thread_start_us":29227,"threads_started":1}
I20260811 09:08:32.730552 8495 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44752:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 09:08:32.738519 8495 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 09:08:32.776461 10888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:32.760419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.129:34797 (local address 127.0.0.1:37784)
0811 09:08:32.762205 (+ 1786us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:32.762223 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:32.762383 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:32.763465 (+ 1082us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:32.763471 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:32.763500 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:08:32.764358 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:32.764376 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:32.768792 (+ 4416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:32.768799 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:32.771507 (+ 2708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:32.771529 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:32.775612 (+ 4083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:32.775620 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:32.775820 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:32.775855 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 09:08:32.775930 (+ 75us) client_negotiation.cc:240] Negotiation successful
0811 09:08:32.776089 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1621,"thread_start_us":1511,"threads_started":1}
I20260811 09:08:32.776507 10889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:32.760678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37784 (local address 127.28.154.129:34797)
0811 09:08:32.761577 (+ 899us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:32.761587 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:32.762522 (+ 935us) server_negotiation.cc:407] Connection header received
0811 09:08:32.762685 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:32.762694 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:32.763013 (+ 319us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:32.763222 (+ 209us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:08:32.764620 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:32.766630 (+ 2010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:32.773869 (+ 7239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:32.775365 (+ 1496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:32.775571 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:32.775772 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:32.776014 (+ 242us) server_negotiation.cc:299] Negotiation successful
0811 09:08:32.776190 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":709,"thread_start_us":1367,"threads_started":1}
I20260811 09:08:32.780738 10176 tablet_service.cc:1467] Processing CreateTablet for tablet 886c5c4c12ba4dd58d957ac145d115f8 (DEFAULT_TABLE table=test-table [id=356cf0876cb64609a88f4e7c0381b098]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:08:32.782339 10886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:32.759687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.130:35019 (local address 127.0.0.1:49050)
0811 09:08:32.760819 (+ 1132us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:32.760841 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:32.760989 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:32.774879 (+ 13890us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:32.774884 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:32.774917 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:08:32.775511 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:32.775526 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:32.777968 (+ 2442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:32.777974 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:32.780094 (+ 2120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:32.780108 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:32.781666 (+ 1558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:32.781671 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:32.781814 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:32.781842 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 09:08:32.781899 (+ 57us) client_negotiation.cc:240] Negotiation successful
0811 09:08:32.781994 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":984,"mutex_wait_us":74,"thread_start_us":767,"threads_started":1}
I20260811 09:08:32.782433 10176 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 886c5c4c12ba4dd58d957ac145d115f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:08:32.783339 10887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:32.759947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49050 (local address 127.28.154.130:35019)
0811 09:08:32.774227 (+ 14280us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:32.774236 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:32.774264 (+ 28us) server_negotiation.cc:407] Connection header received
0811 09:08:32.774371 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:32.774379 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:32.774620 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:32.774861 (+ 241us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:08:32.775772 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:32.777785 (+ 2013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:32.780269 (+ 2484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:32.781420 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:32.782773 (+ 1353us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:32.782936 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:32.783008 (+ 72us) server_negotiation.cc:299] Negotiation successful
0811 09:08:32.783135 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14123,"thread_start_us":14038,"threads_started":1}
I20260811 09:08:32.783535 10890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:32.760821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.131:39329 (local address 127.0.0.1:48900)
0811 09:08:32.773452 (+ 12631us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:32.773468 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:32.773607 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:32.775121 (+ 1514us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:32.775125 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:32.775154 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:08:32.775777 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:32.775790 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:32.778058 (+ 2268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:32.778064 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:32.780610 (+ 2546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:32.780625 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:32.782665 (+ 2040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:32.782671 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:32.782858 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:32.782893 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 09:08:32.782964 (+ 71us) client_negotiation.cc:240] Negotiation successful
0811 09:08:32.783093 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12532,"mutex_wait_us":28,"thread_start_us":12424,"threads_started":1}
I20260811 09:08:32.784715 10891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:32.761408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48900 (local address 127.28.154.131:39329)
0811 09:08:32.772933 (+ 11525us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:32.772943 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:32.773586 (+ 643us) server_negotiation.cc:407] Connection header received
0811 09:08:32.773768 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:32.773778 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:32.774632 (+ 854us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:32.774861 (+ 229us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:08:32.776012 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:32.777831 (+ 1819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:32.780802 (+ 2971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:32.782411 (+ 1609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:32.783917 (+ 1506us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:32.784083 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:32.784163 (+ 80us) server_negotiation.cc:299] Negotiation successful
0811 09:08:32.784328 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11285,"thread_start_us":11541,"threads_started":1}
I20260811 09:08:32.787169 10448 tablet_service.cc:1467] Processing CreateTablet for tablet 886c5c4c12ba4dd58d957ac145d115f8 (DEFAULT_TABLE table=test-table [id=356cf0876cb64609a88f4e7c0381b098]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:08:32.788777 10448 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 886c5c4c12ba4dd58d957ac145d115f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:08:32.788581 10716 tablet_service.cc:1467] Processing CreateTablet for tablet 886c5c4c12ba4dd58d957ac145d115f8 (DEFAULT_TABLE table=test-table [id=356cf0876cb64609a88f4e7c0381b098]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:08:32.789927 10716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 886c5c4c12ba4dd58d957ac145d115f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:08:32.796773 10893 tablet_bootstrap.cc:492] T 886c5c4c12ba4dd58d957ac145d115f8 P 2ed5f79097f84d8ea5faca70a459d1ca: Bootstrap starting.
I20260811 09:08:32.801697 10893 tablet_bootstrap.cc:654] T 886c5c4c12ba4dd58d957ac145d115f8 P 2ed5f79097f84d8ea5faca70a459d1ca: Neither blocks nor log segments found. Creating new log.
I20260811 09:08:32.803260 10893 log.cc:824] T 886c5c4c12ba4dd58d957ac145d115f8 P 2ed5f79097f84d8ea5faca70a459d1ca: Log is configured to *not* fsync() on all Append() calls
I20260811 09:08:32.806531 10896 tablet_bootstrap.cc:492] T 886c5c4c12ba4dd58d957ac145d115f8 P c62d163d6d3a435b821da93a68c125f9: Bootstrap starting.
I20260811 09:08:32.809556 10904 tablet_bootstrap.cc:492] T 886c5c4c12ba4dd58d957ac145d115f8 P 5675cf4144864bd9aa773a78d2e2ed50: Bootstrap starting.
I20260811 09:08:32.811156 10896 tablet_bootstrap.cc:654] T 886c5c4c12ba4dd58d957ac145d115f8 P c62d163d6d3a435b821da93a68c125f9: Neither blocks nor log segments found. Creating new log.
I20260811 09:08:32.812745 10896 log.cc:824] T 886c5c4c12ba4dd58d957ac145d115f8 P c62d163d6d3a435b821da93a68c125f9: Log is configured to *not* fsync() on all Append() calls
I20260811 09:08:32.813475 10904 tablet_bootstrap.cc:654] T 886c5c4c12ba4dd58d957ac145d115f8 P 5675cf4144864bd9aa773a78d2e2ed50: Neither blocks nor log segments found. Creating new log.
I20260811 09:08:32.814714 10893 tablet_bootstrap.cc:492] T 886c5c4c12ba4dd58d957ac145d115f8 P 2ed5f79097f84d8ea5faca70a459d1ca: No bootstrap required, opened a new log
I20260811 09:08:32.814740 10904 log.cc:824] T 886c5c4c12ba4dd58d957ac145d115f8 P 5675cf4144864bd9aa773a78d2e2ed50: Log is configured to *not* fsync() on all Append() calls
I20260811 09:08:32.815054 10893 ts_tablet_manager.cc:1397] T 886c5c4c12ba4dd58d957ac145d115f8 P 2ed5f79097f84d8ea5faca70a459d1ca: Time spent bootstrapping tablet: real 0.019s user 0.009s sys 0.005s
I20260811 09:08:32.816118 10896 tablet_bootstrap.cc:492] T 886c5c4c12ba4dd58d957ac145d115f8 P c62d163d6d3a435b821da93a68c125f9: No bootstrap required, opened a new log
I20260811 09:08:32.816573 10896 ts_tablet_manager.cc:1397] T 886c5c4c12ba4dd58d957ac145d115f8 P c62d163d6d3a435b821da93a68c125f9: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.007s
I20260811 09:08:32.817397 10904 tablet_bootstrap.cc:492] T 886c5c4c12ba4dd58d957ac145d115f8 P 5675cf4144864bd9aa773a78d2e2ed50: No bootstrap required, opened a new log
I20260811 09:08:32.817654 10904 ts_tablet_manager.cc:1397] T 886c5c4c12ba4dd58d957ac145d115f8 P 5675cf4144864bd9aa773a78d2e2ed50: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260811 09:08:32.825659 10904 raft_consensus.cc:348] T 886c5c4c12ba4dd58d957ac145d115f8 P 5675cf4144864bd9aa773a78d2e2ed50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5675cf4144864bd9aa773a78d2e2ed50" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 35019 } } peers { permanent_uuid: "c62d163d6d3a435b821da93a68c125f9" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 39329 } } peers { permanent_uuid: "2ed5f79097f84d8ea5faca70a459d1ca" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 34797 } }
I20260811 09:08:32.826134 10904 raft_consensus.cc:374] T 886c5c4c12ba4dd58d957ac145d115f8 P 5675cf4144864bd9aa773a78d2e2ed50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:08:32.826256 10904 raft_consensus.cc:729] T 886c5c4c12ba4dd58d957ac145d115f8 P 5675cf4144864bd9aa773a78d2e2ed50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5675cf4144864bd9aa773a78d2e2ed50, State: Initialized, Role: FOLLOWER
I20260811 09:08:32.826396 10893 raft_consensus.cc:348] T 886c5c4c12ba4dd58d957ac145d115f8 P 2ed5f79097f84d8ea5faca70a459d1ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5675cf4144864bd9aa773a78d2e2ed50" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 35019 } } peers { permanent_uuid: "c62d163d6d3a435b821da93a68c125f9" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 39329 } } peers { permanent_uuid: "2ed5f79097f84d8ea5faca70a459d1ca" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 34797 } }
I20260811 09:08:32.826839 10893 raft_consensus.cc:374] T 886c5c4c12ba4dd58d957ac145d115f8 P 2ed5f79097f84d8ea5faca70a459d1ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:08:32.826961 10893 raft_consensus.cc:729] T 886c5c4c12ba4dd58d957ac145d115f8 P 2ed5f79097f84d8ea5faca70a459d1ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ed5f79097f84d8ea5faca70a459d1ca, State: Initialized, Role: FOLLOWER
I20260811 09:08:32.826958 10904 consensus_queue.cc:260] T 886c5c4c12ba4dd58d957ac145d115f8 P 5675cf4144864bd9aa773a78d2e2ed50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5675cf4144864bd9aa773a78d2e2ed50" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 35019 } } peers { permanent_uuid: "c62d163d6d3a435b821da93a68c125f9" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 39329 } } peers { permanent_uuid: "2ed5f79097f84d8ea5faca70a459d1ca" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 34797 } }
I20260811 09:08:32.827409 10896 raft_consensus.cc:348] T 886c5c4c12ba4dd58d957ac145d115f8 P c62d163d6d3a435b821da93a68c125f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5675cf4144864bd9aa773a78d2e2ed50" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 35019 } } peers { permanent_uuid: "c62d163d6d3a435b821da93a68c125f9" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 39329 } } peers { permanent_uuid: "2ed5f79097f84d8ea5faca70a459d1ca" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 34797 } }
I20260811 09:08:32.827565 10893 consensus_queue.cc:260] T 886c5c4c12ba4dd58d957ac145d115f8 P 2ed5f79097f84d8ea5faca70a459d1ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5675cf4144864bd9aa773a78d2e2ed50" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 35019 } } peers { permanent_uuid: "c62d163d6d3a435b821da93a68c125f9" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 39329 } } peers { permanent_uuid: "2ed5f79097f84d8ea5faca70a459d1ca" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 34797 } }
I20260811 09:08:32.827869 10896 raft_consensus.cc:374] T 886c5c4c12ba4dd58d957ac145d115f8 P c62d163d6d3a435b821da93a68c125f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:08:32.828001 10896 raft_consensus.cc:729] T 886c5c4c12ba4dd58d957ac145d115f8 P c62d163d6d3a435b821da93a68c125f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c62d163d6d3a435b821da93a68c125f9, State: Initialized, Role: FOLLOWER
I20260811 09:08:32.828758 10896 consensus_queue.cc:260] T 886c5c4c12ba4dd58d957ac145d115f8 P c62d163d6d3a435b821da93a68c125f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5675cf4144864bd9aa773a78d2e2ed50" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 35019 } } peers { permanent_uuid: "c62d163d6d3a435b821da93a68c125f9" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 39329 } } peers { permanent_uuid: "2ed5f79097f84d8ea5faca70a459d1ca" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 34797 } }
I20260811 09:08:32.829602 10893 ts_tablet_manager.cc:1428] T 886c5c4c12ba4dd58d957ac145d115f8 P 2ed5f79097f84d8ea5faca70a459d1ca: Time spent starting tablet: real 0.014s user 0.010s sys 0.005s
I20260811 09:08:32.837785 10896 ts_tablet_manager.cc:1428] T 886c5c4c12ba4dd58d957ac145d115f8 P c62d163d6d3a435b821da93a68c125f9: Time spent starting tablet: real 0.021s user 0.017s sys 0.003s
I20260811 09:08:32.838179 10833 heartbeater.cc:503] Master 127.28.154.190:42507 was elected leader, sending a full tablet report...
I20260811 09:08:32.838411 10904 ts_tablet_manager.cc:1428] T 886c5c4c12ba4dd58d957ac145d115f8 P 5675cf4144864bd9aa773a78d2e2ed50: Time spent starting tablet: real 0.021s user 0.018s sys 0.000s
I20260811 09:08:32.841223 10570 heartbeater.cc:503] Master 127.28.154.190:42507 was elected leader, sending a full tablet report...
I20260811 09:08:32.844050 10910 raft_consensus.cc:482] T 886c5c4c12ba4dd58d957ac145d115f8 P c62d163d6d3a435b821da93a68c125f9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:08:32.844277 10910 raft_consensus.cc:504] T 886c5c4c12ba4dd58d957ac145d115f8 P c62d163d6d3a435b821da93a68c125f9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5675cf4144864bd9aa773a78d2e2ed50" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 35019 } } peers { permanent_uuid: "c62d163d6d3a435b821da93a68c125f9" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 39329 } } peers { permanent_uuid: "2ed5f79097f84d8ea5faca70a459d1ca" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 34797 } }
I20260811 09:08:32.845881 10910 leader_election.cc:288] T 886c5c4c12ba4dd58d957ac145d115f8 P c62d163d6d3a435b821da93a68c125f9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5675cf4144864bd9aa773a78d2e2ed50 (127.28.154.130:35019), 2ed5f79097f84d8ea5faca70a459d1ca (127.28.154.129:34797)
I20260811 09:08:32.851822 10889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:32.846432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.131:40389 (local address 127.28.154.129:34797)
0811 09:08:32.846606 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:32.846610 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:32.846624 (+ 14us) server_negotiation.cc:407] Connection header received
0811 09:08:32.846676 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:32.846679 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:32.846730 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:32.846806 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:08:32.847207 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:32.847964 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:32.849776 (+ 1812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:32.850647 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:32.850786 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:32.850857 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:32.851408 (+ 551us) server_negotiation.cc:299] Negotiation successful
0811 09:08:32.851538 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 09:08:32.851840 10842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:32.846087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.129:34797 (local address 127.28.154.131:40389)
0811 09:08:32.846362 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:32.846379 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:32.846522 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:32.846849 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:32.846854 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:32.846882 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:08:32.847078 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:32.847087 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:32.848124 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:32.848130 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:32.849594 (+ 1464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:32.849607 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:32.851032 (+ 1425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:32.851039 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:32.851254 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:32.851283 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 09:08:32.851362 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 09:08:32.851517 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"mutex_wait_us":118}
I20260811 09:08:32.852018 10887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:32.846250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.131:54773 (local address 127.28.154.130:35019)
0811 09:08:32.846444 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:32.846450 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:32.846469 (+ 19us) server_negotiation.cc:407] Connection header received
0811 09:08:32.846527 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:32.846531 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:32.846598 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:32.846679 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:08:32.847105 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:32.847838 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:32.849702 (+ 1864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:32.850711 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:32.850854 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:32.850938 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:32.851613 (+ 675us) server_negotiation.cc:299] Negotiation successful
0811 09:08:32.851747 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 09:08:32.852504 10845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:32.846087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.130:35019 (local address 127.28.154.131:54773)
0811 09:08:32.846240 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:32.846259 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:32.846370 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:32.846714 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:32.846719 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:32.846745 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:08:32.846957 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:32.846967 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:32.847981 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:32.847987 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:32.849533 (+ 1546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:32.849546 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:32.850900 (+ 1354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:32.850906 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:32.851119 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:32.851148 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 09:08:32.851232 (+ 84us) client_negotiation.cc:240] Negotiation successful
0811 09:08:32.851388 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37,"mutex_wait_us":53}
I20260811 09:08:32.853138 10213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "886c5c4c12ba4dd58d957ac145d115f8" candidate_uuid: "c62d163d6d3a435b821da93a68c125f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ed5f79097f84d8ea5faca70a459d1ca" is_pre_election: true
I20260811 09:08:32.853474 10487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "886c5c4c12ba4dd58d957ac145d115f8" candidate_uuid: "c62d163d6d3a435b821da93a68c125f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5675cf4144864bd9aa773a78d2e2ed50" is_pre_election: true
I20260811 09:08:32.853667 10213 raft_consensus.cc:2454] T 886c5c4c12ba4dd58d957ac145d115f8 P 2ed5f79097f84d8ea5faca70a459d1ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c62d163d6d3a435b821da93a68c125f9 in term 0.
I20260811 09:08:32.853927 10487 raft_consensus.cc:2454] T 886c5c4c12ba4dd58d957ac145d115f8 P 5675cf4144864bd9aa773a78d2e2ed50 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c62d163d6d3a435b821da93a68c125f9 in term 0.
I20260811 09:08:32.854467 10623 leader_election.cc:302] T 886c5c4c12ba4dd58d957ac145d115f8 P c62d163d6d3a435b821da93a68c125f9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ed5f79097f84d8ea5faca70a459d1ca, c62d163d6d3a435b821da93a68c125f9; no voters:
I20260811 09:08:32.854939 10910 raft_consensus.cc:2781] T 886c5c4c12ba4dd58d957ac145d115f8 P c62d163d6d3a435b821da93a68c125f9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 09:08:32.855100 10910 raft_consensus.cc:482] T 886c5c4c12ba4dd58d957ac145d115f8 P c62d163d6d3a435b821da93a68c125f9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 09:08:32.855202 10910 raft_consensus.cc:3037] T 886c5c4c12ba4dd58d957ac145d115f8 P c62d163d6d3a435b821da93a68c125f9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:08:32.857607 10910 raft_consensus.cc:504] T 886c5c4c12ba4dd58d957ac145d115f8 P c62d163d6d3a435b821da93a68c125f9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5675cf4144864bd9aa773a78d2e2ed50" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 35019 } } peers { permanent_uuid: "c62d163d6d3a435b821da93a68c125f9" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 39329 } } peers { permanent_uuid: "2ed5f79097f84d8ea5faca70a459d1ca" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 34797 } }
I20260811 09:08:32.858212 10910 leader_election.cc:288] T 886c5c4c12ba4dd58d957ac145d115f8 P c62d163d6d3a435b821da93a68c125f9 [CANDIDATE]: Term 1 election: Requested vote from peers 5675cf4144864bd9aa773a78d2e2ed50 (127.28.154.130:35019), 2ed5f79097f84d8ea5faca70a459d1ca (127.28.154.129:34797)
I20260811 09:08:32.858732 10487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "886c5c4c12ba4dd58d957ac145d115f8" candidate_uuid: "c62d163d6d3a435b821da93a68c125f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5675cf4144864bd9aa773a78d2e2ed50"
I20260811 09:08:32.858866 10213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "886c5c4c12ba4dd58d957ac145d115f8" candidate_uuid: "c62d163d6d3a435b821da93a68c125f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ed5f79097f84d8ea5faca70a459d1ca"
I20260811 09:08:32.858944 10487 raft_consensus.cc:3037] T 886c5c4c12ba4dd58d957ac145d115f8 P 5675cf4144864bd9aa773a78d2e2ed50 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:08:32.859114 10213 raft_consensus.cc:3037] T 886c5c4c12ba4dd58d957ac145d115f8 P 2ed5f79097f84d8ea5faca70a459d1ca [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:08:32.861385 10487 raft_consensus.cc:2454] T 886c5c4c12ba4dd58d957ac145d115f8 P 5675cf4144864bd9aa773a78d2e2ed50 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c62d163d6d3a435b821da93a68c125f9 in term 1.
I20260811 09:08:32.861465 10213 raft_consensus.cc:2454] T 886c5c4c12ba4dd58d957ac145d115f8 P 2ed5f79097f84d8ea5faca70a459d1ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c62d163d6d3a435b821da93a68c125f9 in term 1.
I20260811 09:08:32.861817 10622 leader_election.cc:302] T 886c5c4c12ba4dd58d957ac145d115f8 P c62d163d6d3a435b821da93a68c125f9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5675cf4144864bd9aa773a78d2e2ed50, c62d163d6d3a435b821da93a68c125f9; no voters:
I20260811 09:08:32.862134 10910 raft_consensus.cc:2781] T 886c5c4c12ba4dd58d957ac145d115f8 P c62d163d6d3a435b821da93a68c125f9 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:08:32.862690 10910 raft_consensus.cc:686] T 886c5c4c12ba4dd58d957ac145d115f8 P c62d163d6d3a435b821da93a68c125f9 [term 1 LEADER]: Becoming Leader. State: Replica: c62d163d6d3a435b821da93a68c125f9, State: Running, Role: LEADER
I20260811 09:08:32.863257 10910 consensus_queue.cc:237] T 886c5c4c12ba4dd58d957ac145d115f8 P c62d163d6d3a435b821da93a68c125f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5675cf4144864bd9aa773a78d2e2ed50" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 35019 } } peers { permanent_uuid: "c62d163d6d3a435b821da93a68c125f9" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 39329 } } peers { permanent_uuid: "2ed5f79097f84d8ea5faca70a459d1ca" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 34797 } }
I20260811 09:08:32.868568 8495 catalog_manager.cc:5495] T 886c5c4c12ba4dd58d957ac145d115f8 P c62d163d6d3a435b821da93a68c125f9 reported cstate change: term changed from 0 to 1, leader changed from <none> to c62d163d6d3a435b821da93a68c125f9 (127.28.154.131). New cstate: current_term: 1 leader_uuid: "c62d163d6d3a435b821da93a68c125f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5675cf4144864bd9aa773a78d2e2ed50" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 35019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c62d163d6d3a435b821da93a68c125f9" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 39329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2ed5f79097f84d8ea5faca70a459d1ca" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 34797 } health_report { overall_health: UNKNOWN } } }
W20260811 09:08:32.875885 10836 tablet.cc:2369] T 886c5c4c12ba4dd58d957ac145d115f8 P c62d163d6d3a435b821da93a68c125f9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 09:08:32.882579 10301 tablet.cc:2369] T 886c5c4c12ba4dd58d957ac145d115f8 P 2ed5f79097f84d8ea5faca70a459d1ca: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 09:08:32.908639 10891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:32.903030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48908 (local address 127.28.154.131:39329)
0811 09:08:32.903224 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:32.903229 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:32.903257 (+ 28us) server_negotiation.cc:407] Connection header received
0811 09:08:32.903403 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:32.903407 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:32.903494 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:32.903619 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:08:32.903990 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:32.904865 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:32.906309 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:32.906881 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:32.906981 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:32.907483 (+ 502us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:08:32.908023 (+ 540us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:08:32.908149 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:32.908251 (+ 102us) server_negotiation.cc:299] Negotiation successful
0811 09:08:32.908319 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 09:08:32.917222 10213 raft_consensus.cc:1261] T 886c5c4c12ba4dd58d957ac145d115f8 P 2ed5f79097f84d8ea5faca70a459d1ca [term 1 FOLLOWER]: Refusing update from remote peer c62d163d6d3a435b821da93a68c125f9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:08:32.917241 10487 raft_consensus.cc:1261] T 886c5c4c12ba4dd58d957ac145d115f8 P 5675cf4144864bd9aa773a78d2e2ed50 [term 1 FOLLOWER]: Refusing update from remote peer c62d163d6d3a435b821da93a68c125f9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:08:32.918190 10911 consensus_queue.cc:1035] T 886c5c4c12ba4dd58d957ac145d115f8 P c62d163d6d3a435b821da93a68c125f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ed5f79097f84d8ea5faca70a459d1ca" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 34797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:08:32.918577 10910 consensus_queue.cc:1035] T 886c5c4c12ba4dd58d957ac145d115f8 P c62d163d6d3a435b821da93a68c125f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5675cf4144864bd9aa773a78d2e2ed50" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 35019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:08:32.930230 10916 mvcc.cc:205] Tried to move back new op lower bound from 7317255425698525184 to 7317255425491722240. Current Snapshot: MvccSnapshot[applied={T|T < 7317255425698525184}]
I20260811 09:08:32.938750 10919 mvcc.cc:205] Tried to move back new op lower bound from 7317255425698525184 to 7317255425491722240. Current Snapshot: MvccSnapshot[applied={T|T < 7317255425698525184}]
I20260811 09:08:32.952863 8495 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44752:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:08:32.953135 8495 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44752:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:08:32.959678 8495 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 41f9d60d712e4276b0da79549a2090e5: Sending DeleteTablet for 3 replicas of tablet 886c5c4c12ba4dd58d957ac145d115f8
I20260811 09:08:32.961217 10448 tablet_service.cc:1514] Processing DeleteTablet for tablet 886c5c4c12ba4dd58d957ac145d115f8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:08:32 UTC) from {username='oryx', principal='oryx/127.28.154.190@KRBTEST.COM'} at 127.0.0.1:49050
I20260811 09:08:32.961285 10716 tablet_service.cc:1514] Processing DeleteTablet for tablet 886c5c4c12ba4dd58d957ac145d115f8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:08:32 UTC) from {username='oryx', principal='oryx/127.28.154.190@KRBTEST.COM'} at 127.0.0.1:48900
I20260811 09:08:32.961664 10176 tablet_service.cc:1514] Processing DeleteTablet for tablet 886c5c4c12ba4dd58d957ac145d115f8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:08:32 UTC) from {username='oryx', principal='oryx/127.28.154.190@KRBTEST.COM'} at 127.0.0.1:37784
I20260811 09:08:32.973124 10940 tablet_replica.cc:330] T 886c5c4c12ba4dd58d957ac145d115f8 P 2ed5f79097f84d8ea5faca70a459d1ca: stopping tablet replica
I20260811 09:08:32.973664 10940 raft_consensus.cc:2229] T 886c5c4c12ba4dd58d957ac145d115f8 P 2ed5f79097f84d8ea5faca70a459d1ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:08:32.974288 10940 raft_consensus.cc:2258] T 886c5c4c12ba4dd58d957ac145d115f8 P 2ed5f79097f84d8ea5faca70a459d1ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:08:32.980674 10935 tablet_replica.cc:330] T 886c5c4c12ba4dd58d957ac145d115f8 P 5675cf4144864bd9aa773a78d2e2ed50: stopping tablet replica
I20260811 09:08:32.981213 10935 raft_consensus.cc:2229] T 886c5c4c12ba4dd58d957ac145d115f8 P 5675cf4144864bd9aa773a78d2e2ed50 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:08:32.982160 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 8445
I20260811 09:08:32.983352 8546 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:08:32.983778 10935 raft_consensus.cc:2258] T 886c5c4c12ba4dd58d957ac145d115f8 P 5675cf4144864bd9aa773a78d2e2ed50 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:08:32.986545 10936 tablet_replica.cc:330] T 886c5c4c12ba4dd58d957ac145d115f8 P c62d163d6d3a435b821da93a68c125f9: stopping tablet replica
I20260811 09:08:32.988636 10936 raft_consensus.cc:2229] T 886c5c4c12ba4dd58d957ac145d115f8 P c62d163d6d3a435b821da93a68c125f9 [term 1 LEADER]: Raft consensus shutting down.
I20260811 09:08:32.988812 10940 ts_tablet_manager.cc:1905] T 886c5c4c12ba4dd58d957ac145d115f8 P 2ed5f79097f84d8ea5faca70a459d1ca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:08:32.993577 10936 raft_consensus.cc:2258] T 886c5c4c12ba4dd58d957ac145d115f8 P c62d163d6d3a435b821da93a68c125f9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:08:32.994204 10935 ts_tablet_manager.cc:1905] T 886c5c4c12ba4dd58d957ac145d115f8 P 5675cf4144864bd9aa773a78d2e2ed50: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:08:32.997555 10940 ts_tablet_manager.cc:1918] T 886c5c4c12ba4dd58d957ac145d115f8 P 2ed5f79097f84d8ea5faca70a459d1ca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:08:32.997836 10940 log.cc:1196] T 886c5c4c12ba4dd58d957ac145d115f8 P 2ed5f79097f84d8ea5faca70a459d1ca: Deleting WAL directory at /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/ts-0/wal/wals/886c5c4c12ba4dd58d957ac145d115f8
I20260811 09:08:32.998555 10940 ts_tablet_manager.cc:1939] T 886c5c4c12ba4dd58d957ac145d115f8 P 2ed5f79097f84d8ea5faca70a459d1ca: Deleting consensus metadata
I20260811 09:08:33.001796 10936 ts_tablet_manager.cc:1905] T 886c5c4c12ba4dd58d957ac145d115f8 P c62d163d6d3a435b821da93a68c125f9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:08:33.002884 10935 ts_tablet_manager.cc:1918] T 886c5c4c12ba4dd58d957ac145d115f8 P 5675cf4144864bd9aa773a78d2e2ed50: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:08:33.003093 10935 log.cc:1196] T 886c5c4c12ba4dd58d957ac145d115f8 P 5675cf4144864bd9aa773a78d2e2ed50: Deleting WAL directory at /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/ts-1/wal/wals/886c5c4c12ba4dd58d957ac145d115f8
I20260811 09:08:33.003756 10935 ts_tablet_manager.cc:1939] T 886c5c4c12ba4dd58d957ac145d115f8 P 5675cf4144864bd9aa773a78d2e2ed50: Deleting consensus metadata
I20260811 09:08:33.012123 10936 ts_tablet_manager.cc:1918] T 886c5c4c12ba4dd58d957ac145d115f8 P c62d163d6d3a435b821da93a68c125f9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:08:33.012413 10936 log.cc:1196] T 886c5c4c12ba4dd58d957ac145d115f8 P c62d163d6d3a435b821da93a68c125f9: Deleting WAL directory at /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/ts-2/wal/wals/886c5c4c12ba4dd58d957ac145d115f8
I20260811 09:08:33.013115 10936 ts_tablet_manager.cc:1939] T 886c5c4c12ba4dd58d957ac145d115f8 P c62d163d6d3a435b821da93a68c125f9: Deleting consensus metadata
I20260811 09:08:33.017346 10840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:32.965125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36136 (local address 127.28.154.189:33355)
0811 09:08:32.965327 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:32.965335 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:32.975330 (+ 9995us) server_negotiation.cc:407] Connection header received
0811 09:08:32.975608 (+ 278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:32.975614 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:32.975697 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:32.975764 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:32.980110 (+ 4346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:32.981365 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:32.984556 (+ 3191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:32.985259 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:32.985415 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:33.005550 (+ 20135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:33.005568 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:33.005573 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:08:33.005627 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:33.009747 (+ 4120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:33.012508 (+ 2761us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:33.012517 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:33.012521 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:33.012769 (+ 248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:33.013546 (+ 777us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:33.013553 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:33.013556 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:33.014004 (+ 448us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:33.014285 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:33.016660 (+ 2375us) server_negotiation.cc:299] Negotiation successful
0811 09:08:33.016949 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 09:08:33.039583 10864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:32.980292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49428 (local address 127.28.154.188:37317)
0811 09:08:32.980497 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:32.980503 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:33.017538 (+ 37035us) server_negotiation.cc:407] Connection header received
0811 09:08:33.017688 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:33.017694 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:33.017846 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:33.017949 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:33.020608 (+ 2659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:33.021436 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:33.023509 (+ 2073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:33.024462 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:33.024610 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:33.032062 (+ 7452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:33.032071 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:33.032076 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:08:33.032125 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:33.036250 (+ 4125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:33.037264 (+ 1014us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:33.037270 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:33.037274 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:33.037449 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:33.037912 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:33.037918 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:33.037922 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:33.038298 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:33.038564 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:33.038999 (+ 435us) server_negotiation.cc:299] Negotiation successful
0811 09:08:33.039254 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 09:08:33.137610 8473 catalog_manager.cc:4841] TS 2ed5f79097f84d8ea5faca70a459d1ca (127.28.154.129:34797): tablet 886c5c4c12ba4dd58d957ac145d115f8 (table test-table [id=356cf0876cb64609a88f4e7c0381b098]) successfully deleted
I20260811 09:08:33.138279 8472 catalog_manager.cc:4841] TS c62d163d6d3a435b821da93a68c125f9 (127.28.154.131:39329): tablet 886c5c4c12ba4dd58d957ac145d115f8 (table test-table [id=356cf0876cb64609a88f4e7c0381b098]) successfully deleted
I20260811 09:08:33.138583 8472 catalog_manager.cc:4841] TS 5675cf4144864bd9aa773a78d2e2ed50 (127.28.154.130:35019): tablet 886c5c4c12ba4dd58d957ac145d115f8 (table test-table [id=356cf0876cb64609a88f4e7c0381b098]) successfully deleted
I20260811 09:08:33.138897 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 8445
I20260811 09:08:33.215124 10840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:33.181781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36148 (local address 127.28.154.189:33355)
0811 09:08:33.182003 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:33.182009 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:33.198757 (+ 16748us) server_negotiation.cc:407] Connection header received
0811 09:08:33.199112 (+ 355us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:33.199118 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:33.199209 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:33.199282 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:33.201592 (+ 2310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:33.202349 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:33.204323 (+ 1974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:33.205004 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:33.205128 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:33.209363 (+ 4235us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:33.209369 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:33.209374 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:08:33.209415 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:33.212447 (+ 3032us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:33.213120 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:33.213126 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:33.213130 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:33.213285 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:33.213660 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:33.213666 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:33.213670 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:33.213980 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:33.214226 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:33.214562 (+ 336us) server_negotiation.cc:299] Negotiation successful
0811 09:08:33.214795 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 09:08:33.216002 10864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:33.181685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49436 (local address 127.28.154.188:37317)
0811 09:08:33.181872 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:33.181877 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:33.198213 (+ 16336us) server_negotiation.cc:407] Connection header received
0811 09:08:33.198684 (+ 471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:33.198691 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:33.198809 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:33.198888 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:33.201428 (+ 2540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:33.202537 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:33.204414 (+ 1877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:33.204858 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:33.204974 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:33.208565 (+ 3591us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:33.208573 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:33.208577 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:08:33.208621 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:33.212664 (+ 4043us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:33.213603 (+ 939us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:33.213608 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:33.213612 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:33.213818 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:33.214275 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:33.214280 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:33.214282 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:33.214636 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:33.214866 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:33.215455 (+ 589us) server_negotiation.cc:299] Negotiation successful
0811 09:08:33.215693 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 09:08:33.943567 10957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:33.942654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:42507 (local address 127.28.154.131:44591)
0811 09:08:33.943173 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:33.943333 (+ 160us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:42507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":227,"threads_started":1}
W20260811 09:08:33.944168 10833 heartbeater.cc:644] Failed to heartbeat to 127.28.154.190:42507 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.28.154.190:42507: connect: Connection refused (error 111)
I20260811 09:08:33.944900 10960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:33.944126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:42507 (local address 127.28.154.130:35683)
0811 09:08:33.944582 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:33.944701 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:42507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":237,"threads_started":1}
W20260811 09:08:33.945369 10570 heartbeater.cc:644] Failed to heartbeat to 127.28.154.190:42507 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.28.154.190:42507: connect: Connection refused (error 111)
I20260811 09:08:33.949417 10962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:33.948768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:42507 (local address 127.28.154.129:37253)
0811 09:08:33.949168 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:33.949276 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:42507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":203,"threads_started":1}
W20260811 09:08:33.949904 10292 heartbeater.cc:644] Failed to heartbeat to 127.28.154.190:42507 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.28.154.190:42507: connect: Connection refused (error 111)
I20260811 09:08:34.459939 10968 raft_consensus.cc:482] T 00000000000000000000000000000000 P 169951c4d2524ade9f73f60b2d109e3a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 41f9d60d712e4276b0da79549a2090e5)
I20260811 09:08:34.460219 10968 raft_consensus.cc:504] T 00000000000000000000000000000000 P 169951c4d2524ade9f73f60b2d109e3a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "41f9d60d712e4276b0da79549a2090e5" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 42507 } } peers { permanent_uuid: "169951c4d2524ade9f73f60b2d109e3a" member_type: VOTER last_known_addr { host: "127.28.154.189" port: 33355 } attrs { promote: false } } peers { permanent_uuid: "c6402a22cf5b441bbde21194d27f3d49" member_type: VOTER last_known_addr { host: "127.28.154.188" port: 37317 } attrs { promote: false } }
I20260811 09:08:34.461362 10969 raft_consensus.cc:482] T 00000000000000000000000000000000 P c6402a22cf5b441bbde21194d27f3d49 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 41f9d60d712e4276b0da79549a2090e5)
I20260811 09:08:34.461561 10969 raft_consensus.cc:504] T 00000000000000000000000000000000 P c6402a22cf5b441bbde21194d27f3d49 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "41f9d60d712e4276b0da79549a2090e5" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 42507 } } peers { permanent_uuid: "169951c4d2524ade9f73f60b2d109e3a" member_type: VOTER last_known_addr { host: "127.28.154.189" port: 33355 } attrs { promote: false } } peers { permanent_uuid: "c6402a22cf5b441bbde21194d27f3d49" member_type: VOTER last_known_addr { host: "127.28.154.188" port: 37317 } attrs { promote: false } }
I20260811 09:08:34.461782 10968 leader_election.cc:288] T 00000000000000000000000000000000 P 169951c4d2524ade9f73f60b2d109e3a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 41f9d60d712e4276b0da79549a2090e5 (127.28.154.190:42507), c6402a22cf5b441bbde21194d27f3d49 (127.28.154.188:37317)
I20260811 09:08:34.463094 10971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:34.461991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:42507 (local address 127.0.0.1:45158)
0811 09:08:34.462548 (+ 557us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:34.462805 (+ 257us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:42507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":416,"thread_start_us":440,"threads_started":1}
W20260811 09:08:34.463455 9605 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.28.154.190:42507: connect: Connection refused (error 111)
I20260811 09:08:34.472276 10969 leader_election.cc:288] T 00000000000000000000000000000000 P c6402a22cf5b441bbde21194d27f3d49 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 41f9d60d712e4276b0da79549a2090e5 (127.28.154.190:42507), 169951c4d2524ade9f73f60b2d109e3a (127.28.154.189:33355)
I20260811 09:08:34.473205 10972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:34.471801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:42507 (local address 127.0.0.1:45176)
0811 09:08:34.472007 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:34.472217 (+ 210us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:42507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260811 09:08:34.473417 10978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:34.462516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:42507 (local address 127.0.0.1:45172)
0811 09:08:34.472906 (+ 10390us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:34.473137 (+ 231us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:42507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":10270,"thread_start_us":10357,"threads_started":1}
W20260811 09:08:34.473403 9605 leader_election.cc:334] T 00000000000000000000000000000000 P 169951c4d2524ade9f73f60b2d109e3a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 41f9d60d712e4276b0da79549a2090e5 (127.28.154.190:42507): Network error: Client connection negotiation failed: client connection to 127.28.154.190:42507: connect: Connection refused (error 111)
I20260811 09:08:34.473925 10971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:34.462006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.188:37317 (local address 127.0.0.1:49448)
0811 09:08:34.463304 (+ 1298us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:34.463323 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:34.463483 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:34.464329 (+ 846us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:34.464362 (+ 33us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:34.464406 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:08:34.464689 (+ 283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:34.464702 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:34.466545 (+ 1843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:34.466552 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:34.469012 (+ 2460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:34.469033 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:34.473125 (+ 4092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:34.473132 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:34.473344 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:34.473387 (+ 43us) client_negotiation.cc:769] Sending connection context
0811 09:08:34.473487 (+ 100us) client_negotiation.cc:240] Negotiation successful
0811 09:08:34.473660 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1207,"thread_start_us":9911,"threads_started":1}
W20260811 09:08:34.474041 9959 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.28.154.190:42507: connect: Connection refused (error 111)
I20260811 09:08:34.474699 10974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:34.462226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49448 (local address 127.28.154.188:37317)
0811 09:08:34.463984 (+ 1758us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:34.463991 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:34.464010 (+ 19us) server_negotiation.cc:407] Connection header received
0811 09:08:34.464084 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:34.464090 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:34.464165 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:34.464839 (+ 674us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:08:34.465061 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:34.466359 (+ 1298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:34.471596 (+ 5237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:34.472906 (+ 1310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:34.473675 (+ 769us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:34.473749 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:34.473801 (+ 52us) server_negotiation.cc:299] Negotiation successful
0811 09:08:34.473933 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1637,"thread_start_us":1712,"threads_started":1}
I20260811 09:08:34.475451 10003 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "169951c4d2524ade9f73f60b2d109e3a" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "c6402a22cf5b441bbde21194d27f3d49" is_pre_election: true
I20260811 09:08:34.475926 10003 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c6402a22cf5b441bbde21194d27f3d49 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 169951c4d2524ade9f73f60b2d109e3a in term 1.
I20260811 09:08:34.476118 10978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:34.475730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:42507 (local address 127.0.0.1:45190)
0811 09:08:34.475920 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:34.475990 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:42507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
W20260811 09:08:34.476317 9959 leader_election.cc:334] T 00000000000000000000000000000000 P c6402a22cf5b441bbde21194d27f3d49 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 41f9d60d712e4276b0da79549a2090e5 (127.28.154.190:42507): Network error: Client connection negotiation failed: client connection to 127.28.154.190:42507: connect: Connection refused (error 111)
I20260811 09:08:34.476537 9607 leader_election.cc:302] T 00000000000000000000000000000000 P 169951c4d2524ade9f73f60b2d109e3a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 169951c4d2524ade9f73f60b2d109e3a, c6402a22cf5b441bbde21194d27f3d49; no voters: 41f9d60d712e4276b0da79549a2090e5
I20260811 09:08:34.476959 10968 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 169951c4d2524ade9f73f60b2d109e3a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260811 09:08:34.477094 10968 raft_consensus.cc:482] T 00000000000000000000000000000000 P 169951c4d2524ade9f73f60b2d109e3a [term 1 FOLLOWER]: Starting leader election (detected failure of leader 41f9d60d712e4276b0da79549a2090e5)
I20260811 09:08:34.477193 10968 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 169951c4d2524ade9f73f60b2d109e3a [term 1 FOLLOWER]: Advancing to term 2
I20260811 09:08:34.479720 10968 raft_consensus.cc:504] T 00000000000000000000000000000000 P 169951c4d2524ade9f73f60b2d109e3a [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "41f9d60d712e4276b0da79549a2090e5" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 42507 } } peers { permanent_uuid: "169951c4d2524ade9f73f60b2d109e3a" member_type: VOTER last_known_addr { host: "127.28.154.189" port: 33355 } attrs { promote: false } } peers { permanent_uuid: "c6402a22cf5b441bbde21194d27f3d49" member_type: VOTER last_known_addr { host: "127.28.154.188" port: 37317 } attrs { promote: false } }
I20260811 09:08:34.480407 10979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:34.469221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36162 (local address 127.28.154.189:33355)
0811 09:08:34.471623 (+ 2402us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:34.471630 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:34.473378 (+ 1748us) server_negotiation.cc:407] Connection header received
0811 09:08:34.473656 (+ 278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:34.473663 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:34.473798 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:34.473902 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:08:34.474910 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:34.476135 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:34.478258 (+ 2123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:34.479320 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:34.479507 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:34.479585 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:34.479852 (+ 267us) server_negotiation.cc:299] Negotiation successful
0811 09:08:34.479992 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2247,"thread_start_us":2008,"threads_started":1}
I20260811 09:08:34.480304 10975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:34.468870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.189:33355 (local address 127.0.0.1:36162)
0811 09:08:34.472973 (+ 4103us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:34.472995 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:34.473160 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:34.474252 (+ 1092us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:34.474255 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:34.474282 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:08:34.474653 (+ 371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:34.474667 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:34.476314 (+ 1647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:34.476320 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:34.478053 (+ 1733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:34.478083 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:34.479505 (+ 1422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:34.479511 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:34.479693 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:34.479718 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 09:08:34.479805 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 09:08:34.479947 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3977,"thread_start_us":3775,"threads_started":1}
I20260811 09:08:34.480724 10968 leader_election.cc:288] T 00000000000000000000000000000000 P 169951c4d2524ade9f73f60b2d109e3a [CANDIDATE]: Term 2 election: Requested vote from peers 41f9d60d712e4276b0da79549a2090e5 (127.28.154.190:42507), c6402a22cf5b441bbde21194d27f3d49 (127.28.154.188:37317)
I20260811 09:08:34.481021 10003 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "169951c4d2524ade9f73f60b2d109e3a" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "c6402a22cf5b441bbde21194d27f3d49"
I20260811 09:08:34.481272 10003 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c6402a22cf5b441bbde21194d27f3d49 [term 1 FOLLOWER]: Advancing to term 2
I20260811 09:08:34.481280 9656 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "c6402a22cf5b441bbde21194d27f3d49" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "169951c4d2524ade9f73f60b2d109e3a" is_pre_election: true
I20260811 09:08:34.481382 10971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:34.480617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:42507 (local address 127.0.0.1:45194)
0811 09:08:34.481136 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:34.481229 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:42507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":405}
I20260811 09:08:34.481705 9656 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 169951c4d2524ade9f73f60b2d109e3a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c6402a22cf5b441bbde21194d27f3d49 in current term 2: Already voted for candidate 169951c4d2524ade9f73f60b2d109e3a in this term.
I20260811 09:08:34.482281 9964 leader_election.cc:302] T 00000000000000000000000000000000 P c6402a22cf5b441bbde21194d27f3d49 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c6402a22cf5b441bbde21194d27f3d49; no voters: 169951c4d2524ade9f73f60b2d109e3a, 41f9d60d712e4276b0da79549a2090e5
I20260811 09:08:34.482553 10971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:34.482188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:42507 (local address 127.0.0.1:45196)
0811 09:08:34.482330 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:34.482420 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:42507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260811 09:08:34.482744 9605 leader_election.cc:334] T 00000000000000000000000000000000 P 169951c4d2524ade9f73f60b2d109e3a [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 41f9d60d712e4276b0da79549a2090e5 (127.28.154.190:42507): Network error: Client connection negotiation failed: client connection to 127.28.154.190:42507: connect: Connection refused (error 111)
I20260811 09:08:34.484135 10003 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c6402a22cf5b441bbde21194d27f3d49 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 169951c4d2524ade9f73f60b2d109e3a in term 2.
I20260811 09:08:34.484309 10969 raft_consensus.cc:2726] T 00000000000000000000000000000000 P c6402a22cf5b441bbde21194d27f3d49 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 09:08:34.484572 9607 leader_election.cc:302] T 00000000000000000000000000000000 P 169951c4d2524ade9f73f60b2d109e3a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 169951c4d2524ade9f73f60b2d109e3a, c6402a22cf5b441bbde21194d27f3d49; no voters: 41f9d60d712e4276b0da79549a2090e5
I20260811 09:08:34.484889 10968 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 169951c4d2524ade9f73f60b2d109e3a [term 2 FOLLOWER]: Leader election won for term 2
I20260811 09:08:34.485380 10968 raft_consensus.cc:686] T 00000000000000000000000000000000 P 169951c4d2524ade9f73f60b2d109e3a [term 2 LEADER]: Becoming Leader. State: Replica: 169951c4d2524ade9f73f60b2d109e3a, State: Running, Role: LEADER
I20260811 09:08:34.485786 10968 consensus_queue.cc:237] T 00000000000000000000000000000000 P 169951c4d2524ade9f73f60b2d109e3a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "41f9d60d712e4276b0da79549a2090e5" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 42507 } } peers { permanent_uuid: "169951c4d2524ade9f73f60b2d109e3a" member_type: VOTER last_known_addr { host: "127.28.154.189" port: 33355 } attrs { promote: false } } peers { permanent_uuid: "c6402a22cf5b441bbde21194d27f3d49" member_type: VOTER last_known_addr { host: "127.28.154.188" port: 37317 } attrs { promote: false } }
I20260811 09:08:34.487962 10982 sys_catalog.cc:455] T 00000000000000000000000000000000 P 169951c4d2524ade9f73f60b2d109e3a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 169951c4d2524ade9f73f60b2d109e3a. Latest consensus state: current_term: 2 leader_uuid: "169951c4d2524ade9f73f60b2d109e3a" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "41f9d60d712e4276b0da79549a2090e5" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 42507 } } peers { permanent_uuid: "169951c4d2524ade9f73f60b2d109e3a" member_type: VOTER last_known_addr { host: "127.28.154.189" port: 33355 } attrs { promote: false } } peers { permanent_uuid: "c6402a22cf5b441bbde21194d27f3d49" member_type: VOTER last_known_addr { host: "127.28.154.188" port: 37317 } attrs { promote: false } } }
I20260811 09:08:34.488396 10982 sys_catalog.cc:458] T 00000000000000000000000000000000 P 169951c4d2524ade9f73f60b2d109e3a [sys.catalog]: This master's current role is: LEADER
I20260811 09:08:34.489116 10984 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:08:34.494395 10984 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:08:34.495582 10984 catalog_manager.cc:1259] Loaded cluster ID: 039df53890a0413897b986c5c7822102
I20260811 09:08:34.495677 10984 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:08:34.496942 10984 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:08:34.497816 10984 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 169951c4d2524ade9f73f60b2d109e3a: Loaded TSK: 0
I20260811 09:08:34.498807 10984 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 09:08:34.552227 9636 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36148:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 09:08:34.559350 9636 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 09:08:34.563655 10971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:34.563180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:42507 (local address 127.0.0.1:45220)
0811 09:08:34.563363 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:34.563467 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:42507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260811 09:08:34.563956 10003 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c6402a22cf5b441bbde21194d27f3d49 [term 2 FOLLOWER]: Refusing update from remote peer 169951c4d2524ade9f73f60b2d109e3a: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260811 09:08:34.564769 10971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:34.564416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:42507 (local address 127.0.0.1:45226)
0811 09:08:34.564557 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:34.564648 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:42507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260811 09:08:34.564718 10982 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 169951c4d2524ade9f73f60b2d109e3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c6402a22cf5b441bbde21194d27f3d49" member_type: VOTER last_known_addr { host: "127.28.154.188" port: 37317 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260811 09:08:34.564991 9605 consensus_peers.cc:487] T 00000000000000000000000000000000 P 169951c4d2524ade9f73f60b2d109e3a -> Peer 41f9d60d712e4276b0da79549a2090e5 (127.28.154.190:42507): Couldn't send request to peer 41f9d60d712e4276b0da79549a2090e5. Status: Network error: Client connection negotiation failed: client connection to 127.28.154.190:42507: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 09:08:34.567452 10969 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6402a22cf5b441bbde21194d27f3d49 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 169951c4d2524ade9f73f60b2d109e3a. Latest consensus state: current_term: 2 leader_uuid: "169951c4d2524ade9f73f60b2d109e3a" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "41f9d60d712e4276b0da79549a2090e5" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 42507 } } peers { permanent_uuid: "169951c4d2524ade9f73f60b2d109e3a" member_type: VOTER last_known_addr { host: "127.28.154.189" port: 33355 } attrs { promote: false } } peers { permanent_uuid: "c6402a22cf5b441bbde21194d27f3d49" member_type: VOTER last_known_addr { host: "127.28.154.188" port: 37317 } attrs { promote: false } } }
I20260811 09:08:34.567718 10969 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6402a22cf5b441bbde21194d27f3d49 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 09:08:34.569489 10968 sys_catalog.cc:455] T 00000000000000000000000000000000 P 169951c4d2524ade9f73f60b2d109e3a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "169951c4d2524ade9f73f60b2d109e3a" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "41f9d60d712e4276b0da79549a2090e5" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 42507 } } peers { permanent_uuid: "169951c4d2524ade9f73f60b2d109e3a" member_type: VOTER last_known_addr { host: "127.28.154.189" port: 33355 } attrs { promote: false } } peers { permanent_uuid: "c6402a22cf5b441bbde21194d27f3d49" member_type: VOTER last_known_addr { host: "127.28.154.188" port: 37317 } attrs { promote: false } } }
I20260811 09:08:34.569818 10968 sys_catalog.cc:458] T 00000000000000000000000000000000 P 169951c4d2524ade9f73f60b2d109e3a [sys.catalog]: This master's current role is: LEADER
I20260811 09:08:34.570516 10969 sys_catalog.cc:455] T 00000000000000000000000000000000 P c6402a22cf5b441bbde21194d27f3d49 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "169951c4d2524ade9f73f60b2d109e3a" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "41f9d60d712e4276b0da79549a2090e5" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 42507 } } peers { permanent_uuid: "169951c4d2524ade9f73f60b2d109e3a" member_type: VOTER last_known_addr { host: "127.28.154.189" port: 33355 } attrs { promote: false } } peers { permanent_uuid: "c6402a22cf5b441bbde21194d27f3d49" member_type: VOTER last_known_addr { host: "127.28.154.188" port: 37317 } attrs { promote: false } } }
I20260811 09:08:34.570791 10969 sys_catalog.cc:458] T 00000000000000000000000000000000 P c6402a22cf5b441bbde21194d27f3d49 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 09:08:34.588223 10997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:34.579353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.129:34797 (local address 127.0.0.1:37790)
0811 09:08:34.580243 (+ 890us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:34.580261 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:34.580431 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:34.580979 (+ 548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:34.580985 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:34.581011 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:08:34.581364 (+ 353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:34.581375 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:34.582955 (+ 1580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:34.582960 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:34.585270 (+ 2310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:34.585292 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:34.587121 (+ 1829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:34.587129 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:34.587388 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:34.587434 (+ 46us) client_negotiation.cc:769] Sending connection context
0811 09:08:34.587558 (+ 124us) client_negotiation.cc:240] Negotiation successful
0811 09:08:34.587725 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":754,"thread_start_us":610,"threads_started":1}
I20260811 09:08:34.589680 10998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:34.579494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37790 (local address 127.28.154.129:34797)
0811 09:08:34.580237 (+ 743us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:34.580243 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:34.580354 (+ 111us) server_negotiation.cc:407] Connection header received
0811 09:08:34.580615 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:34.580623 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:34.580705 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:34.580796 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:08:34.581559 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:34.582813 (+ 1254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:34.585486 (+ 2673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:34.586849 (+ 1363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:34.587046 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:34.587133 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:34.587599 (+ 466us) server_negotiation.cc:299] Negotiation successful
0811 09:08:34.587768 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":402,"thread_start_us":270,"threads_started":1}
I20260811 09:08:34.590684 10176 tablet_service.cc:1467] Processing CreateTablet for tablet 3f0cdcb660d2436fa0aac83fc31a82b5 (DEFAULT_TABLE table=test-table [id=c5720b2869bf48f9bf43b8fcf553ec68]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:08:34.591244 10176 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f0cdcb660d2436fa0aac83fc31a82b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:08:34.596953 10999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:34.579653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49054 (local address 127.28.154.130:35019)
0811 09:08:34.590838 (+ 11185us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:34.590844 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:34.590862 (+ 18us) server_negotiation.cc:407] Connection header received
0811 09:08:34.590944 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:34.590950 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:34.591033 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:34.591137 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:08:34.591664 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:34.592438 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:34.594862 (+ 2424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:34.595881 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:34.596082 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:34.596163 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:34.596526 (+ 363us) server_negotiation.cc:299] Negotiation successful
0811 09:08:34.596672 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11042,"thread_start_us":11601,"threads_started":1}
I20260811 09:08:34.596987 10972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:34.578937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.130:35019 (local address 127.0.0.1:49054)
0811 09:08:34.579291 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:34.579309 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:34.579422 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:34.591179 (+ 11757us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:34.591184 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:34.591217 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:08:34.591501 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:34.591518 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:34.592641 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:34.592646 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:34.594632 (+ 1986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:34.594647 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:34.596151 (+ 1504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:34.596159 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:34.596320 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:34.596388 (+ 68us) client_negotiation.cc:769] Sending connection context
0811 09:08:34.596483 (+ 95us) client_negotiation.cc:240] Negotiation successful
0811 09:08:34.596635 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"mutex_wait_us":1}
I20260811 09:08:34.597991 10448 tablet_service.cc:1467] Processing CreateTablet for tablet 3f0cdcb660d2436fa0aac83fc31a82b5 (DEFAULT_TABLE table=test-table [id=c5720b2869bf48f9bf43b8fcf553ec68]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:08:34.598434 10448 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f0cdcb660d2436fa0aac83fc31a82b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:08:34.598748 11001 tablet_bootstrap.cc:492] T 3f0cdcb660d2436fa0aac83fc31a82b5 P 2ed5f79097f84d8ea5faca70a459d1ca: Bootstrap starting.
I20260811 09:08:34.599068 10971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:34.578552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.131:39329 (local address 127.0.0.1:48912)
0811 09:08:34.578743 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:34.578764 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:34.578879 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:34.593115 (+ 14236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:34.593121 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:34.593152 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:08:34.593398 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:34.593411 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:34.594809 (+ 1398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:34.594815 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:34.596847 (+ 2032us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:34.596864 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:34.598457 (+ 1593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:34.598463 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:34.598639 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:34.598666 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 09:08:34.598720 (+ 54us) client_negotiation.cc:240] Negotiation successful
0811 09:08:34.598816 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260811 09:08:34.600822 11001 tablet_bootstrap.cc:654] T 3f0cdcb660d2436fa0aac83fc31a82b5 P 2ed5f79097f84d8ea5faca70a459d1ca: Neither blocks nor log segments found. Creating new log.
I20260811 09:08:34.600951 10996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:34.578638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48912 (local address 127.28.154.131:39329)
0811 09:08:34.592783 (+ 14145us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:34.592790 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:34.592811 (+ 21us) server_negotiation.cc:407] Connection header received
0811 09:08:34.592880 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:34.592885 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:34.592967 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:34.593071 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:08:34.593575 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:34.594651 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:34.597086 (+ 2435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:34.598270 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:34.599415 (+ 1145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:34.599500 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:34.599566 (+ 66us) server_negotiation.cc:299] Negotiation successful
0811 09:08:34.599718 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14018,"thread_start_us":13871,"threads_started":1}
I20260811 09:08:34.601627 10716 tablet_service.cc:1467] Processing CreateTablet for tablet 3f0cdcb660d2436fa0aac83fc31a82b5 (DEFAULT_TABLE table=test-table [id=c5720b2869bf48f9bf43b8fcf553ec68]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:08:34.601996 10716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f0cdcb660d2436fa0aac83fc31a82b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:08:34.603479 11001 tablet_bootstrap.cc:492] T 3f0cdcb660d2436fa0aac83fc31a82b5 P 2ed5f79097f84d8ea5faca70a459d1ca: No bootstrap required, opened a new log
I20260811 09:08:34.603686 11001 ts_tablet_manager.cc:1397] T 3f0cdcb660d2436fa0aac83fc31a82b5 P 2ed5f79097f84d8ea5faca70a459d1ca: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 09:08:34.604720 11001 raft_consensus.cc:348] T 3f0cdcb660d2436fa0aac83fc31a82b5 P 2ed5f79097f84d8ea5faca70a459d1ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62d163d6d3a435b821da93a68c125f9" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 39329 } } peers { permanent_uuid: "5675cf4144864bd9aa773a78d2e2ed50" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 35019 } } peers { permanent_uuid: "2ed5f79097f84d8ea5faca70a459d1ca" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 34797 } }
I20260811 09:08:34.605119 11001 raft_consensus.cc:374] T 3f0cdcb660d2436fa0aac83fc31a82b5 P 2ed5f79097f84d8ea5faca70a459d1ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:08:34.605296 11001 raft_consensus.cc:729] T 3f0cdcb660d2436fa0aac83fc31a82b5 P 2ed5f79097f84d8ea5faca70a459d1ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ed5f79097f84d8ea5faca70a459d1ca, State: Initialized, Role: FOLLOWER
I20260811 09:08:34.605829 11001 consensus_queue.cc:260] T 3f0cdcb660d2436fa0aac83fc31a82b5 P 2ed5f79097f84d8ea5faca70a459d1ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62d163d6d3a435b821da93a68c125f9" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 39329 } } peers { permanent_uuid: "5675cf4144864bd9aa773a78d2e2ed50" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 35019 } } peers { permanent_uuid: "2ed5f79097f84d8ea5faca70a459d1ca" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 34797 } }
I20260811 09:08:34.609140 11010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:34.607918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:42507 (local address 127.28.154.129:34683)
0811 09:08:34.608675 (+ 757us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:34.608884 (+ 209us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:42507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":608,"threads_started":1}
I20260811 09:08:34.610226 11001 ts_tablet_manager.cc:1428] T 3f0cdcb660d2436fa0aac83fc31a82b5 P 2ed5f79097f84d8ea5faca70a459d1ca: Time spent starting tablet: real 0.006s user 0.003s sys 0.000s
I20260811 09:08:34.611312 10296 heartbeater.cc:503] Master 127.28.154.189:33355 was elected leader, sending a full tablet report...
I20260811 09:08:34.611322 11010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:34.610922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:42507 (local address 127.28.154.129:47295)
0811 09:08:34.611064 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:34.611156 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:42507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260811 09:08:34.613252 11008 tablet_bootstrap.cc:492] T 3f0cdcb660d2436fa0aac83fc31a82b5 P 5675cf4144864bd9aa773a78d2e2ed50: Bootstrap starting.
I20260811 09:08:34.614861 11008 tablet_bootstrap.cc:654] T 3f0cdcb660d2436fa0aac83fc31a82b5 P 5675cf4144864bd9aa773a78d2e2ed50: Neither blocks nor log segments found. Creating new log.
I20260811 09:08:34.617237 11008 tablet_bootstrap.cc:492] T 3f0cdcb660d2436fa0aac83fc31a82b5 P 5675cf4144864bd9aa773a78d2e2ed50: No bootstrap required, opened a new log
I20260811 09:08:34.617388 11008 ts_tablet_manager.cc:1397] T 3f0cdcb660d2436fa0aac83fc31a82b5 P 5675cf4144864bd9aa773a78d2e2ed50: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260811 09:08:34.618039 11008 raft_consensus.cc:348] T 3f0cdcb660d2436fa0aac83fc31a82b5 P 5675cf4144864bd9aa773a78d2e2ed50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62d163d6d3a435b821da93a68c125f9" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 39329 } } peers { permanent_uuid: "5675cf4144864bd9aa773a78d2e2ed50" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 35019 } } peers { permanent_uuid: "2ed5f79097f84d8ea5faca70a459d1ca" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 34797 } }
I20260811 09:08:34.618228 11008 raft_consensus.cc:374] T 3f0cdcb660d2436fa0aac83fc31a82b5 P 5675cf4144864bd9aa773a78d2e2ed50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:08:34.618323 11008 raft_consensus.cc:729] T 3f0cdcb660d2436fa0aac83fc31a82b5 P 5675cf4144864bd9aa773a78d2e2ed50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5675cf4144864bd9aa773a78d2e2ed50, State: Initialized, Role: FOLLOWER
I20260811 09:08:34.618571 11008 consensus_queue.cc:260] T 3f0cdcb660d2436fa0aac83fc31a82b5 P 5675cf4144864bd9aa773a78d2e2ed50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62d163d6d3a435b821da93a68c125f9" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 39329 } } peers { permanent_uuid: "5675cf4144864bd9aa773a78d2e2ed50" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 35019 } } peers { permanent_uuid: "2ed5f79097f84d8ea5faca70a459d1ca" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 34797 } }
I20260811 09:08:34.619822 11008 ts_tablet_manager.cc:1428] T 3f0cdcb660d2436fa0aac83fc31a82b5 P 5675cf4144864bd9aa773a78d2e2ed50: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260811 09:08:34.622128 10572 heartbeater.cc:503] Master 127.28.154.189:33355 was elected leader, sending a full tablet report...
I20260811 09:08:34.624418 11016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:34.619817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:42507 (local address 127.28.154.130:43457)
0811 09:08:34.624003 (+ 4186us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:34.624157 (+ 154us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:42507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4082,"thread_start_us":3839,"threads_started":1}
I20260811 09:08:34.625587 11016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:34.625159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:42507 (local address 127.28.154.130:35103)
0811 09:08:34.625321 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:34.625441 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:42507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260811 09:08:34.626421 11012 tablet_bootstrap.cc:492] T 3f0cdcb660d2436fa0aac83fc31a82b5 P c62d163d6d3a435b821da93a68c125f9: Bootstrap starting.
I20260811 09:08:34.628264 11012 tablet_bootstrap.cc:654] T 3f0cdcb660d2436fa0aac83fc31a82b5 P c62d163d6d3a435b821da93a68c125f9: Neither blocks nor log segments found. Creating new log.
I20260811 09:08:34.630487 11012 tablet_bootstrap.cc:492] T 3f0cdcb660d2436fa0aac83fc31a82b5 P c62d163d6d3a435b821da93a68c125f9: No bootstrap required, opened a new log
I20260811 09:08:34.630633 11012 ts_tablet_manager.cc:1397] T 3f0cdcb660d2436fa0aac83fc31a82b5 P c62d163d6d3a435b821da93a68c125f9: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260811 09:08:34.631304 11012 raft_consensus.cc:348] T 3f0cdcb660d2436fa0aac83fc31a82b5 P c62d163d6d3a435b821da93a68c125f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62d163d6d3a435b821da93a68c125f9" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 39329 } } peers { permanent_uuid: "5675cf4144864bd9aa773a78d2e2ed50" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 35019 } } peers { permanent_uuid: "2ed5f79097f84d8ea5faca70a459d1ca" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 34797 } }
I20260811 09:08:34.631498 11012 raft_consensus.cc:374] T 3f0cdcb660d2436fa0aac83fc31a82b5 P c62d163d6d3a435b821da93a68c125f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:08:34.631590 11012 raft_consensus.cc:729] T 3f0cdcb660d2436fa0aac83fc31a82b5 P c62d163d6d3a435b821da93a68c125f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c62d163d6d3a435b821da93a68c125f9, State: Initialized, Role: FOLLOWER
I20260811 09:08:34.631805 11012 consensus_queue.cc:260] T 3f0cdcb660d2436fa0aac83fc31a82b5 P c62d163d6d3a435b821da93a68c125f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62d163d6d3a435b821da93a68c125f9" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 39329 } } peers { permanent_uuid: "5675cf4144864bd9aa773a78d2e2ed50" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 35019 } } peers { permanent_uuid: "2ed5f79097f84d8ea5faca70a459d1ca" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 34797 } }
I20260811 09:08:34.632542 11012 ts_tablet_manager.cc:1428] T 3f0cdcb660d2436fa0aac83fc31a82b5 P c62d163d6d3a435b821da93a68c125f9: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260811 09:08:34.633809 11021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:34.632867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:42507 (local address 127.28.154.131:54513)
0811 09:08:34.633427 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:34.633605 (+ 178us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:42507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":314,"threads_started":1}
I20260811 09:08:34.635016 11021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:34.634584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:42507 (local address 127.28.154.131:45673)
0811 09:08:34.634829 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:34.634898 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:42507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139}
I20260811 09:08:34.635514 10834 heartbeater.cc:503] Master 127.28.154.189:33355 was elected leader, sending a full tablet report...
I20260811 09:08:34.725483 11020 raft_consensus.cc:482] T 3f0cdcb660d2436fa0aac83fc31a82b5 P c62d163d6d3a435b821da93a68c125f9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:08:34.725732 11020 raft_consensus.cc:504] T 3f0cdcb660d2436fa0aac83fc31a82b5 P c62d163d6d3a435b821da93a68c125f9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62d163d6d3a435b821da93a68c125f9" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 39329 } } peers { permanent_uuid: "5675cf4144864bd9aa773a78d2e2ed50" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 35019 } } peers { permanent_uuid: "2ed5f79097f84d8ea5faca70a459d1ca" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 34797 } }
I20260811 09:08:34.726500 11020 leader_election.cc:288] T 3f0cdcb660d2436fa0aac83fc31a82b5 P c62d163d6d3a435b821da93a68c125f9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5675cf4144864bd9aa773a78d2e2ed50 (127.28.154.130:35019), 2ed5f79097f84d8ea5faca70a459d1ca (127.28.154.129:34797)
I20260811 09:08:34.727154 10213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f0cdcb660d2436fa0aac83fc31a82b5" candidate_uuid: "c62d163d6d3a435b821da93a68c125f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ed5f79097f84d8ea5faca70a459d1ca" is_pre_election: true
I20260811 09:08:34.727111 10487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f0cdcb660d2436fa0aac83fc31a82b5" candidate_uuid: "c62d163d6d3a435b821da93a68c125f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5675cf4144864bd9aa773a78d2e2ed50" is_pre_election: true
I20260811 09:08:34.727399 10213 raft_consensus.cc:2454] T 3f0cdcb660d2436fa0aac83fc31a82b5 P 2ed5f79097f84d8ea5faca70a459d1ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c62d163d6d3a435b821da93a68c125f9 in term 0.
I20260811 09:08:34.727634 10487 raft_consensus.cc:2454] T 3f0cdcb660d2436fa0aac83fc31a82b5 P 5675cf4144864bd9aa773a78d2e2ed50 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c62d163d6d3a435b821da93a68c125f9 in term 0.
I20260811 09:08:34.727893 10623 leader_election.cc:302] T 3f0cdcb660d2436fa0aac83fc31a82b5 P c62d163d6d3a435b821da93a68c125f9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ed5f79097f84d8ea5faca70a459d1ca, c62d163d6d3a435b821da93a68c125f9; no voters:
I20260811 09:08:34.728228 11020 raft_consensus.cc:2781] T 3f0cdcb660d2436fa0aac83fc31a82b5 P c62d163d6d3a435b821da93a68c125f9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 09:08:34.728374 11020 raft_consensus.cc:482] T 3f0cdcb660d2436fa0aac83fc31a82b5 P c62d163d6d3a435b821da93a68c125f9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 09:08:34.728470 11020 raft_consensus.cc:3037] T 3f0cdcb660d2436fa0aac83fc31a82b5 P c62d163d6d3a435b821da93a68c125f9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:08:34.730816 11020 raft_consensus.cc:504] T 3f0cdcb660d2436fa0aac83fc31a82b5 P c62d163d6d3a435b821da93a68c125f9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62d163d6d3a435b821da93a68c125f9" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 39329 } } peers { permanent_uuid: "5675cf4144864bd9aa773a78d2e2ed50" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 35019 } } peers { permanent_uuid: "2ed5f79097f84d8ea5faca70a459d1ca" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 34797 } }
I20260811 09:08:34.731429 11020 leader_election.cc:288] T 3f0cdcb660d2436fa0aac83fc31a82b5 P c62d163d6d3a435b821da93a68c125f9 [CANDIDATE]: Term 1 election: Requested vote from peers 5675cf4144864bd9aa773a78d2e2ed50 (127.28.154.130:35019), 2ed5f79097f84d8ea5faca70a459d1ca (127.28.154.129:34797)
I20260811 09:08:34.732107 10213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f0cdcb660d2436fa0aac83fc31a82b5" candidate_uuid: "c62d163d6d3a435b821da93a68c125f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ed5f79097f84d8ea5faca70a459d1ca"
I20260811 09:08:34.732121 10487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f0cdcb660d2436fa0aac83fc31a82b5" candidate_uuid: "c62d163d6d3a435b821da93a68c125f9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5675cf4144864bd9aa773a78d2e2ed50"
I20260811 09:08:34.732328 10213 raft_consensus.cc:3037] T 3f0cdcb660d2436fa0aac83fc31a82b5 P 2ed5f79097f84d8ea5faca70a459d1ca [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:08:34.732378 10487 raft_consensus.cc:3037] T 3f0cdcb660d2436fa0aac83fc31a82b5 P 5675cf4144864bd9aa773a78d2e2ed50 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:08:34.734530 10213 raft_consensus.cc:2454] T 3f0cdcb660d2436fa0aac83fc31a82b5 P 2ed5f79097f84d8ea5faca70a459d1ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c62d163d6d3a435b821da93a68c125f9 in term 1.
I20260811 09:08:34.734567 10487 raft_consensus.cc:2454] T 3f0cdcb660d2436fa0aac83fc31a82b5 P 5675cf4144864bd9aa773a78d2e2ed50 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c62d163d6d3a435b821da93a68c125f9 in term 1.
I20260811 09:08:34.735020 10623 leader_election.cc:302] T 3f0cdcb660d2436fa0aac83fc31a82b5 P c62d163d6d3a435b821da93a68c125f9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2ed5f79097f84d8ea5faca70a459d1ca, 5675cf4144864bd9aa773a78d2e2ed50, c62d163d6d3a435b821da93a68c125f9; no voters:
I20260811 09:08:34.735471 11020 raft_consensus.cc:2781] T 3f0cdcb660d2436fa0aac83fc31a82b5 P c62d163d6d3a435b821da93a68c125f9 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:08:34.735962 11020 raft_consensus.cc:686] T 3f0cdcb660d2436fa0aac83fc31a82b5 P c62d163d6d3a435b821da93a68c125f9 [term 1 LEADER]: Becoming Leader. State: Replica: c62d163d6d3a435b821da93a68c125f9, State: Running, Role: LEADER
I20260811 09:08:34.736259 11020 consensus_queue.cc:237] T 3f0cdcb660d2436fa0aac83fc31a82b5 P c62d163d6d3a435b821da93a68c125f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62d163d6d3a435b821da93a68c125f9" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 39329 } } peers { permanent_uuid: "5675cf4144864bd9aa773a78d2e2ed50" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 35019 } } peers { permanent_uuid: "2ed5f79097f84d8ea5faca70a459d1ca" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 34797 } }
I20260811 09:08:34.737226 11021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:34.736708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:42507 (local address 127.28.154.131:52025)
0811 09:08:34.736925 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:34.737052 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:42507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260811 09:08:34.739619 9636 catalog_manager.cc:5495] T 3f0cdcb660d2436fa0aac83fc31a82b5 P c62d163d6d3a435b821da93a68c125f9 reported cstate change: term changed from 0 to 1, leader changed from <none> to c62d163d6d3a435b821da93a68c125f9 (127.28.154.131). New cstate: current_term: 1 leader_uuid: "c62d163d6d3a435b821da93a68c125f9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c62d163d6d3a435b821da93a68c125f9" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 39329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5675cf4144864bd9aa773a78d2e2ed50" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 35019 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ed5f79097f84d8ea5faca70a459d1ca" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 34797 } health_report { overall_health: UNKNOWN } } }
W20260811 09:08:34.746157 10573 tablet.cc:2369] T 3f0cdcb660d2436fa0aac83fc31a82b5 P 5675cf4144864bd9aa773a78d2e2ed50: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 09:08:34.781268 10996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:34.776632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33766 (local address 127.28.154.131:39329)
0811 09:08:34.776850 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:34.776855 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:34.776874 (+ 19us) server_negotiation.cc:407] Connection header received
0811 09:08:34.776939 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:34.776942 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:34.777008 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:34.777083 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:08:34.777367 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:34.778132 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:34.779565 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:34.779994 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:34.780096 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:34.780497 (+ 401us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:08:34.780607 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:08:34.780689 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:34.780926 (+ 237us) server_negotiation.cc:299] Negotiation successful
0811 09:08:34.781011 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260811 09:08:34.784420 10213 raft_consensus.cc:1261] T 3f0cdcb660d2436fa0aac83fc31a82b5 P 2ed5f79097f84d8ea5faca70a459d1ca [term 1 FOLLOWER]: Refusing update from remote peer c62d163d6d3a435b821da93a68c125f9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:08:34.784479 10487 raft_consensus.cc:1261] T 3f0cdcb660d2436fa0aac83fc31a82b5 P 5675cf4144864bd9aa773a78d2e2ed50 [term 1 FOLLOWER]: Refusing update from remote peer c62d163d6d3a435b821da93a68c125f9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:08:34.785099 11022 consensus_queue.cc:1035] T 3f0cdcb660d2436fa0aac83fc31a82b5 P c62d163d6d3a435b821da93a68c125f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ed5f79097f84d8ea5faca70a459d1ca" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 34797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:08:34.785439 11020 consensus_queue.cc:1035] T 3f0cdcb660d2436fa0aac83fc31a82b5 P c62d163d6d3a435b821da93a68c125f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5675cf4144864bd9aa773a78d2e2ed50" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 35019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:08:34.788054 11010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:34.787455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:42507 (local address 127.28.154.129:50871)
0811 09:08:34.787653 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:34.787789 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:42507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260811 09:08:34.788689 11016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:34.787850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:42507 (local address 127.28.154.130:44091)
0811 09:08:34.788381 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:34.788505 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:42507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":364}
I20260811 09:08:34.799299 11010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:34.795257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:42507 (local address 127.28.154.129:45015)
0811 09:08:34.798596 (+ 3339us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:34.798814 (+ 218us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:42507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":633}
I20260811 09:08:34.800925 11021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:34.799712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:42507 (local address 127.28.154.131:33839)
0811 09:08:34.799906 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:34.800237 (+ 331us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:42507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260811 09:08:34.804100 11021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:34.803601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:42507 (local address 127.28.154.131:45213)
0811 09:08:34.803815 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:34.803905 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:42507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103}
I20260811 09:08:34.805604 11016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:34.804686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:42507 (local address 127.28.154.130:33499)
0811 09:08:34.805316 (+ 630us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:34.805426 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:42507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":517}
I20260811 09:08:34.812578 9636 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36148:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:08:34.812835 9636 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36148:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:08:34.818174 9636 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 169951c4d2524ade9f73f60b2d109e3a: Sending DeleteTablet for 3 replicas of tablet 3f0cdcb660d2436fa0aac83fc31a82b5
I20260811 09:08:34.819468 10176 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f0cdcb660d2436fa0aac83fc31a82b5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:08:34 UTC) from {username='oryx', principal='oryx/127.28.154.189@KRBTEST.COM'} at 127.0.0.1:37790
I20260811 09:08:34.819494 10448 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f0cdcb660d2436fa0aac83fc31a82b5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:08:34 UTC) from {username='oryx', principal='oryx/127.28.154.189@KRBTEST.COM'} at 127.0.0.1:49054
I20260811 09:08:34.819531 10716 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f0cdcb660d2436fa0aac83fc31a82b5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:08:34 UTC) from {username='oryx', principal='oryx/127.28.154.189@KRBTEST.COM'} at 127.0.0.1:48912
I20260811 09:08:34.820664 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 9941
I20260811 09:08:34.834163 11059 tablet_replica.cc:330] T 3f0cdcb660d2436fa0aac83fc31a82b5 P c62d163d6d3a435b821da93a68c125f9: stopping tablet replica
I20260811 09:08:34.834506 11059 raft_consensus.cc:2229] T 3f0cdcb660d2436fa0aac83fc31a82b5 P c62d163d6d3a435b821da93a68c125f9 [term 1 LEADER]: Raft consensus shutting down.
I20260811 09:08:34.835028 11059 raft_consensus.cc:2258] T 3f0cdcb660d2436fa0aac83fc31a82b5 P c62d163d6d3a435b821da93a68c125f9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:08:34.835387 11055 tablet_replica.cc:330] T 3f0cdcb660d2436fa0aac83fc31a82b5 P 5675cf4144864bd9aa773a78d2e2ed50: stopping tablet replica
I20260811 09:08:34.835886 11055 raft_consensus.cc:2229] T 3f0cdcb660d2436fa0aac83fc31a82b5 P 5675cf4144864bd9aa773a78d2e2ed50 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:08:34.836362 11055 raft_consensus.cc:2258] T 3f0cdcb660d2436fa0aac83fc31a82b5 P 5675cf4144864bd9aa773a78d2e2ed50 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:08:34.836393 11059 ts_tablet_manager.cc:1905] T 3f0cdcb660d2436fa0aac83fc31a82b5 P c62d163d6d3a435b821da93a68c125f9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:08:34.837121 11056 tablet_replica.cc:330] T 3f0cdcb660d2436fa0aac83fc31a82b5 P 2ed5f79097f84d8ea5faca70a459d1ca: stopping tablet replica
I20260811 09:08:34.837317 11056 raft_consensus.cc:2229] T 3f0cdcb660d2436fa0aac83fc31a82b5 P 2ed5f79097f84d8ea5faca70a459d1ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:08:34.837641 11056 raft_consensus.cc:2258] T 3f0cdcb660d2436fa0aac83fc31a82b5 P 2ed5f79097f84d8ea5faca70a459d1ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:08:34.838354 11055 ts_tablet_manager.cc:1905] T 3f0cdcb660d2436fa0aac83fc31a82b5 P 5675cf4144864bd9aa773a78d2e2ed50: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:08:34.838763 11056 ts_tablet_manager.cc:1905] T 3f0cdcb660d2436fa0aac83fc31a82b5 P 2ed5f79097f84d8ea5faca70a459d1ca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:08:34.843328 11059 ts_tablet_manager.cc:1918] T 3f0cdcb660d2436fa0aac83fc31a82b5 P c62d163d6d3a435b821da93a68c125f9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:08:34.843538 11059 log.cc:1196] T 3f0cdcb660d2436fa0aac83fc31a82b5 P c62d163d6d3a435b821da93a68c125f9: Deleting WAL directory at /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/ts-2/wal/wals/3f0cdcb660d2436fa0aac83fc31a82b5
I20260811 09:08:34.843997 11059 ts_tablet_manager.cc:1939] T 3f0cdcb660d2436fa0aac83fc31a82b5 P c62d163d6d3a435b821da93a68c125f9: Deleting consensus metadata
I20260811 09:08:34.845374 9605 catalog_manager.cc:4841] TS c62d163d6d3a435b821da93a68c125f9 (127.28.154.131:39329): tablet 3f0cdcb660d2436fa0aac83fc31a82b5 (table test-table [id=c5720b2869bf48f9bf43b8fcf553ec68]) successfully deleted
I20260811 09:08:34.845817 11055 ts_tablet_manager.cc:1918] T 3f0cdcb660d2436fa0aac83fc31a82b5 P 5675cf4144864bd9aa773a78d2e2ed50: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:08:34.845911 11056 ts_tablet_manager.cc:1918] T 3f0cdcb660d2436fa0aac83fc31a82b5 P 2ed5f79097f84d8ea5faca70a459d1ca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:08:34.846004 11055 log.cc:1196] T 3f0cdcb660d2436fa0aac83fc31a82b5 P 5675cf4144864bd9aa773a78d2e2ed50: Deleting WAL directory at /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/ts-1/wal/wals/3f0cdcb660d2436fa0aac83fc31a82b5
I20260811 09:08:34.846046 11056 log.cc:1196] T 3f0cdcb660d2436fa0aac83fc31a82b5 P 2ed5f79097f84d8ea5faca70a459d1ca: Deleting WAL directory at /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786439250192320-29290-0/minicluster-data/ts-0/wal/wals/3f0cdcb660d2436fa0aac83fc31a82b5
I20260811 09:08:34.846385 11055 ts_tablet_manager.cc:1939] T 3f0cdcb660d2436fa0aac83fc31a82b5 P 5675cf4144864bd9aa773a78d2e2ed50: Deleting consensus metadata
I20260811 09:08:34.846501 11056 ts_tablet_manager.cc:1939] T 3f0cdcb660d2436fa0aac83fc31a82b5 P 2ed5f79097f84d8ea5faca70a459d1ca: Deleting consensus metadata
I20260811 09:08:34.847438 9605 catalog_manager.cc:4841] TS 5675cf4144864bd9aa773a78d2e2ed50 (127.28.154.130:35019): tablet 3f0cdcb660d2436fa0aac83fc31a82b5 (table test-table [id=c5720b2869bf48f9bf43b8fcf553ec68]) successfully deleted
I20260811 09:08:34.847587 10998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:34.821418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39658 (local address 127.28.154.129:34797)
0811 09:08:34.821674 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:34.821681 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:34.824808 (+ 3127us) server_negotiation.cc:407] Connection header received
0811 09:08:34.825020 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:34.825026 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:34.825116 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:34.825234 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:34.827065 (+ 1831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:34.828286 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:34.830155 (+ 1869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:34.830879 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:34.831008 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:34.838802 (+ 7794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:34.838836 (+ 34us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:34.838859 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:08:34.838903 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:34.843045 (+ 4142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:34.844087 (+ 1042us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:34.844098 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:34.844102 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:34.844307 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:34.844860 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:34.844866 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:34.844869 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:34.845376 (+ 507us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:34.845643 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:34.846993 (+ 1350us) server_negotiation.cc:299] Negotiation successful
0811 09:08:34.847254 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 09:08:34.847633 9607 catalog_manager.cc:4841] TS 2ed5f79097f84d8ea5faca70a459d1ca (127.28.154.129:34797): tablet 3f0cdcb660d2436fa0aac83fc31a82b5 (table test-table [id=c5720b2869bf48f9bf43b8fcf553ec68]) successfully deleted
I20260811 09:08:34.848225 10280 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:08:34.941838 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 9941
I20260811 09:08:34.969200 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 10335
I20260811 09:08:34.984818 10999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:34.970059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45070 (local address 127.28.154.130:35019)
0811 09:08:34.970273 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:34.970278 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:34.970298 (+ 20us) server_negotiation.cc:407] Connection header received
0811 09:08:34.970420 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:34.970423 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:34.970486 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:34.970581 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:34.972268 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:34.973002 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:34.974740 (+ 1738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:34.975225 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:34.975356 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:34.978485 (+ 3129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:34.978502 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:34.978519 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:08:34.978556 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:34.981777 (+ 3221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:34.982593 (+ 816us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:34.982601 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:34.982603 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:34.982742 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:34.983127 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:34.983132 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:34.983134 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:34.983548 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:34.983802 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:34.984178 (+ 376us) server_negotiation.cc:299] Negotiation successful
0811 09:08:34.984488 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260811 09:08:34.985486 10563 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:08:35.047545 10971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:35.047116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:42507 (local address 127.0.0.1:36288)
0811 09:08:35.047298 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:35.047387 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:42507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260811 09:08:35.048628 10971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:35.048275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:42507 (local address 127.0.0.1:36300)
0811 09:08:35.048426 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:35.048515 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:42507: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260811 09:08:35.088431 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 10335
I20260811 09:08:35.119143 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 10597
I20260811 09:08:35.133322 10996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:35.119946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33770 (local address 127.28.154.131:39329)
0811 09:08:35.120131 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:35.120135 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:35.120156 (+ 21us) server_negotiation.cc:407] Connection header received
0811 09:08:35.120230 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:35.120235 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:35.120304 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:35.120456 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:35.121815 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:35.122907 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:35.124260 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:35.124730 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:35.124957 (+ 227us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:35.127537 (+ 2580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:35.127561 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:35.127581 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:08:35.127618 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:35.130456 (+ 2838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:35.131137 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:35.131145 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:35.131147 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:35.131310 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:35.131699 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:35.131703 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:35.131705 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:35.132089 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:35.132420 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:35.132750 (+ 330us) server_negotiation.cc:299] Negotiation successful
0811 09:08:35.132998 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 09:08:35.133991 10824 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:08:35.256260 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 10597
I20260811 09:08:35.286015 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 9381
I20260811 09:08:35.287154 9693 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:08:35.414052 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 9381
I20260811 09:08:35.447192 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 9566
I20260811 09:08:35.448375 10034 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:08:35.566159 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 9566
2026-08-11T09:08:35Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (10044 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439250192320-29290-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:08:35 dist-test-slave-xt4k krb5kdc[11078](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:08:35 dist-test-slave-xt4k krb5kdc[11078](info): set up 2 sockets
Aug 11 09:08:35 dist-test-slave-xt4k krb5kdc[11078](info): commencing operation
krb5kdc: starting...
W20260811 09:08:36.853343 29290 mini_kdc.cc:121] Time spent starting KDC: real 1.239s user 0.002s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439250192320-29290-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439250192320-29290-0/krb5kdc/test-user.keytab.
Aug 11 09:08:37 dist-test-slave-xt4k krb5kdc[11078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439317, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:08:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:08:37Z Disabled control of system clock
WARNING: no policy specified for oryx/127.28.154.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.28.154.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.28.154.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439250192320-29290-0/krb5kdc/oryx_127.28.154.190.keytab.
Entry for principal oryx/127.28.154.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439250192320-29290-0/krb5kdc/oryx_127.28.154.190.keytab.
I20260811 09:08:37.134254 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439250192320-29290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439250192320-29290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439250192320-29290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.154.190:45067
--webserver_interface=127.28.154.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439250192320-29290-0/krb5kdc/oryx_127.28.154.190.keytab
--principal=oryx/127.28.154.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:39899
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.154.190:45067
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:08:37.503687 11099 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:08:37.504020 11099 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:08:37.504136 11099 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:08:37.512250 11099 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:08:37.512450 11099 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 09:08:37.512528 11099 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:08:37.512627 11099 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:08:37.512697 11099 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:08:37.512784 11099 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:08:37.523213 11099 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:39899
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439250192320-29290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439250192320-29290-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.154.190:45067
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439250192320-29290-0/krb5kdc/oryx_127.28.154.190.keytab
--principal=oryx/127.28.154.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.154.190:45067
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.154.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439250192320-29290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:08:37.524674 11099 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:08:37.526129 11099 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:08:37.538681 11115 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:08:37.538735 11111 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:08:37.538713 11112 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:08:37.539969 11099 server_base.cc:1034] running on GCE node
I20260811 09:08:37.540896 11099 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:08:37.542623 11099 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:08:37.543886 11099 hybrid_clock.cc:648] HybridClock initialized: now 1786439317543807 us; error 50 us; skew 500 ppm
I20260811 09:08:37.544265 11099 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:08:37 dist-test-slave-xt4k krb5kdc[11078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439317, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.154.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:08:37.547847 11099 init.cc:377] Logged in from keytab as oryx/127.28.154.190@KRBTEST.COM (short username oryx)
I20260811 09:08:37.549715 11099 webserver.cc:460] Webserver started at http://127.28.154.190:33485/ using document root <none> and password file <none>
I20260811 09:08:37.550503 11099 fs_manager.cc:359] Metadata directory not provided
I20260811 09:08:37.550614 11099 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:08:37.550941 11099 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:08:37.555052 11099 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439250192320-29290-0/minicluster-data/master-0/data/instance:
uuid: "32c27303c7e1421abd79752385c82cd5"
format_stamp: "Formatted at 2026-08-11 09:08:37 on dist-test-slave-xt4k"
server_key: "f2513cc25ad98ceae8d87c01dd3d9b94"
server_key_iv: "fa8c76b03dfe3115fb7ac91392140b54"
server_key_version: "encryptionkey@0"
I20260811 09:08:37.556252 11099 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439250192320-29290-0/minicluster-data/master-0/wal/instance:
uuid: "32c27303c7e1421abd79752385c82cd5"
format_stamp: "Formatted at 2026-08-11 09:08:37 on dist-test-slave-xt4k"
server_key: "f2513cc25ad98ceae8d87c01dd3d9b94"
server_key_iv: "fa8c76b03dfe3115fb7ac91392140b54"
server_key_version: "encryptionkey@0"
I20260811 09:08:37.562690 11099 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260811 09:08:37.566655 11136 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:08:37.568601 11099 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 09:08:37.568883 11099 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439250192320-29290-0/minicluster-data/master-0/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439250192320-29290-0/minicluster-data/master-0/wal
uuid: "32c27303c7e1421abd79752385c82cd5"
format_stamp: "Formatted at 2026-08-11 09:08:37 on dist-test-slave-xt4k"
server_key: "f2513cc25ad98ceae8d87c01dd3d9b94"
server_key_iv: "fa8c76b03dfe3115fb7ac91392140b54"
server_key_version: "encryptionkey@0"
I20260811 09:08:37.569130 11099 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439250192320-29290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439250192320-29290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439250192320-29290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:08:37.599146 11099 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:08:37.604444 11099 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:08:37.604782 11099 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:08:37.628224 11246 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.190:45067 every 8 connection(s)
I20260811 09:08:37.628218 11099 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.190:45067
I20260811 09:08:37.630272 11099 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
I20260811 09:08:37.634955 11247 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:08:37.635514 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 11099
I20260811 09:08:37.635730 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439250192320-29290-0/minicluster-data/master-0/wal/instance
I20260811 09:08:37.636412 29290 external_mini_cluster.cc:1428] Setting key d87b16e870f3a6c0c2f2562bf717b1be
I20260811 09:08:37.649406 11247 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32c27303c7e1421abd79752385c82cd5: Bootstrap starting.
Aug 11 09:08:37 dist-test-slave-xt4k krb5kdc[11078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439317, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM
I20260811 09:08:37.653386 11247 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 32c27303c7e1421abd79752385c82cd5: Neither blocks nor log segments found. Creating new log.
I20260811 09:08:37.655373 11247 log.cc:824] T 00000000000000000000000000000000 P 32c27303c7e1421abd79752385c82cd5: Log is configured to *not* fsync() on all Append() calls
I20260811 09:08:37.658843 11247 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32c27303c7e1421abd79752385c82cd5: No bootstrap required, opened a new log
I20260811 09:08:37.662961 11255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:37.639168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55702 (local address 127.28.154.190:45067)
0811 09:08:37.640145 (+ 977us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:37.640159 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:37.640207 (+ 48us) server_negotiation.cc:407] Connection header received
0811 09:08:37.642266 (+ 2059us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:37.642295 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:37.642790 (+ 495us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:37.643209 (+ 419us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:37.645289 (+ 2080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:37.647025 (+ 1736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:37.648458 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:37.649182 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:37.649361 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:37.654483 (+ 5122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:37.654493 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:37.654522 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:08:37.654579 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:37.658266 (+ 3687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:37.659098 (+ 832us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:37.659107 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:37.659114 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:37.659312 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:37.659742 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:37.659747 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:37.659749 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:37.660329 (+ 580us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:37.660730 (+ 401us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:37.661270 (+ 540us) server_negotiation.cc:299] Negotiation successful
0811 09:08:37.661777 (+ 507us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":721,"thread_start_us":452,"threads_started":1}
I20260811 09:08:37.667479 11247 raft_consensus.cc:348] T 00000000000000000000000000000000 P 32c27303c7e1421abd79752385c82cd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32c27303c7e1421abd79752385c82cd5" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 45067 } }
I20260811 09:08:37.667943 11247 raft_consensus.cc:374] T 00000000000000000000000000000000 P 32c27303c7e1421abd79752385c82cd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:08:37.668062 11247 raft_consensus.cc:729] T 00000000000000000000000000000000 P 32c27303c7e1421abd79752385c82cd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32c27303c7e1421abd79752385c82cd5, State: Initialized, Role: FOLLOWER
I20260811 09:08:37.668731 11247 consensus_queue.cc:260] T 00000000000000000000000000000000 P 32c27303c7e1421abd79752385c82cd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32c27303c7e1421abd79752385c82cd5" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 45067 } }
I20260811 09:08:37.669026 11247 raft_consensus.cc:388] T 00000000000000000000000000000000 P 32c27303c7e1421abd79752385c82cd5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:08:37.669176 11247 raft_consensus.cc:482] T 00000000000000000000000000000000 P 32c27303c7e1421abd79752385c82cd5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:08:37.669381 11247 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 32c27303c7e1421abd79752385c82cd5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:08:37.672252 11247 raft_consensus.cc:504] T 00000000000000000000000000000000 P 32c27303c7e1421abd79752385c82cd5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32c27303c7e1421abd79752385c82cd5" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 45067 } }
I20260811 09:08:37.672941 11247 leader_election.cc:302] T 00000000000000000000000000000000 P 32c27303c7e1421abd79752385c82cd5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 32c27303c7e1421abd79752385c82cd5; no voters:
I20260811 09:08:37.673678 11247 leader_election.cc:288] T 00000000000000000000000000000000 P 32c27303c7e1421abd79752385c82cd5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:08:37.673883 11257 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 32c27303c7e1421abd79752385c82cd5 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:08:37.674841 11257 raft_consensus.cc:686] T 00000000000000000000000000000000 P 32c27303c7e1421abd79752385c82cd5 [term 1 LEADER]: Becoming Leader. State: Replica: 32c27303c7e1421abd79752385c82cd5, State: Running, Role: LEADER
I20260811 09:08:37.675779 11257 consensus_queue.cc:237] T 00000000000000000000000000000000 P 32c27303c7e1421abd79752385c82cd5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32c27303c7e1421abd79752385c82cd5" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 45067 } }
I20260811 09:08:37.676573 11247 sys_catalog.cc:564] T 00000000000000000000000000000000 P 32c27303c7e1421abd79752385c82cd5 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:08:37.678838 11258 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32c27303c7e1421abd79752385c82cd5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "32c27303c7e1421abd79752385c82cd5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32c27303c7e1421abd79752385c82cd5" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 45067 } } }
I20260811 09:08:37.679530 11258 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32c27303c7e1421abd79752385c82cd5 [sys.catalog]: This master's current role is: LEADER
I20260811 09:08:37.679414 11260 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32c27303c7e1421abd79752385c82cd5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 32c27303c7e1421abd79752385c82cd5. Latest consensus state: current_term: 1 leader_uuid: "32c27303c7e1421abd79752385c82cd5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32c27303c7e1421abd79752385c82cd5" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 45067 } } }
I20260811 09:08:37.679687 11260 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32c27303c7e1421abd79752385c82cd5 [sys.catalog]: This master's current role is: LEADER
I20260811 09:08:37.685153 11265 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:08:37.690629 11265 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:08:37.700623 11265 catalog_manager.cc:1347] Generated new cluster ID: 9ef8d77cbafe49c3b822e667f1ff0115
I20260811 09:08:37.700748 11265 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:08:37.717547 11265 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:08:37.719002 11265 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:08:37.730155 11265 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 32c27303c7e1421abd79752385c82cd5: Generated new TSK 0
I20260811 09:08:37.731376 11265 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.28.154.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.28.154.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.28.154.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439250192320-29290-0/krb5kdc/oryx_127.28.154.129.keytab.
Entry for principal oryx/127.28.154.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439250192320-29290-0/krb5kdc/oryx_127.28.154.129.keytab.
I20260811 09:08:37.819859 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439250192320-29290-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439250192320-29290-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439250192320-29290-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439250192320-29290-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.154.129:0
--local_ip_for_outbound_sockets=127.28.154.129
--webserver_interface=127.28.154.129
--webserver_port=0
--tserver_master_addrs=127.28.154.190:45067
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439250192320-29290-0/krb5kdc/oryx_127.28.154.129.keytab
--principal=oryx/127.28.154.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:39899
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:08:38.176116 11294 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:08:38.176469 11294 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:08:38.176584 11294 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:08:38.184266 11294 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:08:38.184435 11294 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:08:38.184537 11294 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 09:08:38.184623 11294 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.154.129
I20260811 09:08:38.194800 11294 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:39899
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439250192320-29290-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439250192320-29290-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439250192320-29290-0/krb5kdc/oryx_127.28.154.129.keytab
--principal=oryx/127.28.154.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.154.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439250192320-29290-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.154.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.28.154.190:45067
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.154.129
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439250192320-29290-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:08:38.196110 11294 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:08:38.197475 11294 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:08:38.211164 11307 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:08:38.211113 11302 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:08:38.211220 11305 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:08:38.212036 11294 server_base.cc:1034] running on GCE node
I20260811 09:08:38.212960 11294 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:08:38.214057 11294 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:08:38.215332 11294 hybrid_clock.cc:648] HybridClock initialized: now 1786439318215236 us; error 67 us; skew 500 ppm
I20260811 09:08:38.215696 11294 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:08:38 dist-test-slave-xt4k krb5kdc[11078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439318, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.154.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:08:38.218884 11294 init.cc:377] Logged in from keytab as oryx/127.28.154.129@KRBTEST.COM (short username oryx)
I20260811 09:08:38.220446 11294 webserver.cc:460] Webserver started at http://127.28.154.129:33269/ using document root <none> and password file <none>
I20260811 09:08:38.221148 11294 fs_manager.cc:359] Metadata directory not provided
I20260811 09:08:38.221237 11294 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:08:38.221519 11294 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:08:38.224848 11294 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439250192320-29290-0/minicluster-data/ts-0/data/instance:
uuid: "e6ba0c4707a9480b8309f094af83bd48"
format_stamp: "Formatted at 2026-08-11 09:08:38 on dist-test-slave-xt4k"
server_key: "5c91a240ed6013d3e6c15f81786a027b"
server_key_iv: "bb3089cc2b7a34b46928f9a59b109cfa"
server_key_version: "encryptionkey@0"
I20260811 09:08:38.225677 11294 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439250192320-29290-0/minicluster-data/ts-0/wal/instance:
uuid: "e6ba0c4707a9480b8309f094af83bd48"
format_stamp: "Formatted at 2026-08-11 09:08:38 on dist-test-slave-xt4k"
server_key: "5c91a240ed6013d3e6c15f81786a027b"
server_key_iv: "bb3089cc2b7a34b46928f9a59b109cfa"
server_key_version: "encryptionkey@0"
I20260811 09:08:38.231091 11294 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 09:08:38.235098 11318 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:08:38.236801 11294 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 09:08:38.237001 11294 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439250192320-29290-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439250192320-29290-0/minicluster-data/ts-0/wal
uuid: "e6ba0c4707a9480b8309f094af83bd48"
format_stamp: "Formatted at 2026-08-11 09:08:38 on dist-test-slave-xt4k"
server_key: "5c91a240ed6013d3e6c15f81786a027b"
server_key_iv: "bb3089cc2b7a34b46928f9a59b109cfa"
server_key_version: "encryptionkey@0"
I20260811 09:08:38.237241 11294 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439250192320-29290-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439250192320-29290-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439250192320-29290-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:08:38.270994 11294 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:08:38.275785 11294 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:08:38.276067 11294 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:08:38.279302 11294 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:08:38.279400 11294 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:08:38.279538 11294 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:08:38.279606 11294 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 09:08:38 dist-test-slave-xt4k krb5kdc[11078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439318, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.154.129@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM
I20260811 09:08:38.304935 11338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:38.280092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.129:40913 (local address 127.28.154.190:45067)
0811 09:08:38.282593 (+ 2501us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:38.282600 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:38.282624 (+ 24us) server_negotiation.cc:407] Connection header received
0811 09:08:38.284383 (+ 1759us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:38.284389 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:38.284495 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:38.284597 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:38.286858 (+ 2261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:38.288090 (+ 1232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:38.290660 (+ 2570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:38.291400 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:38.291571 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:38.296633 (+ 5062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:38.296641 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:38.296646 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:08:38.296695 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:38.299993 (+ 3298us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:38.301045 (+ 1052us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:38.301051 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:38.301056 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:38.301193 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:38.301798 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:38.301803 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:38.301807 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:38.302214 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:38.302376 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:38.304377 (+ 2001us) server_negotiation.cc:299] Negotiation successful
0811 09:08:38.304635 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2357,"thread_start_us":2157,"threads_started":1}
I20260811 09:08:38.305346 11339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:38.280755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:45067 (local address 127.28.154.129:40913)
0811 09:08:38.281654 (+ 899us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:38.281706 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:38.284051 (+ 2345us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:38.284864 (+ 813us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:38.284873 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:38.285657 (+ 784us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:08:38.286605 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:38.286641 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:38.288325 (+ 1684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:38.288331 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:38.290449 (+ 2118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:38.290476 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:38.291697 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:38.291705 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:38.292003 (+ 298us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:38.292995 (+ 992us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:08:38.293037 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:08:38.296331 (+ 3294us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:08:38.300145 (+ 3814us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:08:38.300155 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:08:38.300178 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:08:38.300894 (+ 716us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:08:38.301316 (+ 422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:08:38.301323 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:08:38.301326 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:08:38.301664 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:08:38.302366 (+ 702us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:08:38.302380 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:08:38.302822 (+ 442us) client_negotiation.cc:769] Sending connection context
0811 09:08:38.303221 (+ 399us) client_negotiation.cc:240] Negotiation successful
0811 09:08:38.303699 (+ 478us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":604,"thread_start_us":371,"threads_started":1}
I20260811 09:08:38.325692 11338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:38.310955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.129:40915 (local address 127.28.154.190:45067)
0811 09:08:38.311103 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:38.311108 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:38.311867 (+ 759us) server_negotiation.cc:407] Connection header received
0811 09:08:38.312012 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:38.312018 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:38.312110 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:38.312222 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:38.313505 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:38.314657 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:38.316609 (+ 1952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:38.317073 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:38.317193 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:38.319100 (+ 1907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:38.319108 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:38.319113 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:08:38.319158 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:38.322578 (+ 3420us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:38.323472 (+ 894us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:38.323479 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:38.323482 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:38.323665 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:38.324095 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:38.324101 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:38.324105 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:38.324532 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:38.324686 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:38.325115 (+ 429us) server_negotiation.cc:299] Negotiation successful
0811 09:08:38.325346 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 09:08:38.325910 11469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:38.311046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:45067 (local address 127.28.154.129:40915)
0811 09:08:38.311754 (+ 708us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:38.311770 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:38.311893 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:38.312234 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:38.312237 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:38.312821 (+ 584us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:08:38.313274 (+ 453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:38.313285 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:38.314806 (+ 1521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:38.314812 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:38.316458 (+ 1646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:38.316480 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:38.317280 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:38.317287 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:38.317465 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:38.318097 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:08:38.318114 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:08:38.318887 (+ 773us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:08:38.322724 (+ 3837us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:08:38.322730 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:08:38.322734 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:08:38.323334 (+ 600us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:08:38.323809 (+ 475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:08:38.323814 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:08:38.323818 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:08:38.323979 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:08:38.324654 (+ 675us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:08:38.324659 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:08:38.324849 (+ 190us) client_negotiation.cc:769] Sending connection context
0811 09:08:38.324988 (+ 139us) client_negotiation.cc:240] Negotiation successful
0811 09:08:38.325404 (+ 416us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":563,"thread_start_us":417,"threads_started":1}
I20260811 09:08:38.326557 11294 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.129:41463
I20260811 09:08:38.326562 11546 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.129:41463 every 8 connection(s)
I20260811 09:08:38.327855 11294 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439250192320-29290-0/minicluster-data/ts-0/data/info.pb
I20260811 09:08:38.329257 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 11294
I20260811 09:08:38.329500 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439250192320-29290-0/minicluster-data/ts-0/wal/instance
I20260811 09:08:38.330324 29290 external_mini_cluster.cc:1428] Setting key 76bb886ac74a39f9cceb75ab52402851
I20260811 09:08:38.332669 11548 heartbeater.cc:347] Connected to a master server at 127.28.154.190:45067
I20260811 09:08:38.332901 11548 heartbeater.cc:464] Registering TS with master...
I20260811 09:08:38.333527 11548 heartbeater.cc:511] Master 127.28.154.190:45067 requested a full tablet report, sending...
I20260811 09:08:38.335525 11167 ts_manager.cc:194] Registered new tserver with Master: e6ba0c4707a9480b8309f094af83bd48 (127.28.154.129:41463)
I20260811 09:08:38.337050 11167 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.28.154.129@KRBTEST.COM'} at 127.28.154.129:40913
WARNING: no policy specified for oryx/127.28.154.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.28.154.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.28.154.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439250192320-29290-0/krb5kdc/oryx_127.28.154.130.keytab.
Entry for principal oryx/127.28.154.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439250192320-29290-0/krb5kdc/oryx_127.28.154.130.keytab.
I20260811 09:08:38.385921 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439250192320-29290-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439250192320-29290-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439250192320-29290-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439250192320-29290-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.154.130:0
--local_ip_for_outbound_sockets=127.28.154.130
--webserver_interface=127.28.154.130
--webserver_port=0
--tserver_master_addrs=127.28.154.190:45067
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439250192320-29290-0/krb5kdc/oryx_127.28.154.130.keytab
--principal=oryx/127.28.154.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:39899
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:08:38.719617 11559 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:08:38.719898 11559 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:08:38.719995 11559 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:08:38.727741 11559 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:08:38.727870 11559 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:08:38.727955 11559 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 09:08:38.728039 11559 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.154.130
I20260811 09:08:38.738770 11559 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:39899
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439250192320-29290-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439250192320-29290-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439250192320-29290-0/krb5kdc/oryx_127.28.154.130.keytab
--principal=oryx/127.28.154.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.154.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439250192320-29290-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.154.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.28.154.190:45067
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.154.130
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439250192320-29290-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:08:38.740096 11559 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:08:38.741477 11559 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:08:38.754036 11574 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:08:38.754019 11572 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:08:38.754015 11569 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:08:38.755173 11559 server_base.cc:1034] running on GCE node
I20260811 09:08:38.755802 11559 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:08:38.756783 11559 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:08:38.758005 11559 hybrid_clock.cc:648] HybridClock initialized: now 1786439318757948 us; error 50 us; skew 500 ppm
I20260811 09:08:38.758315 11559 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:08:38 dist-test-slave-xt4k krb5kdc[11078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439318, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.154.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:08:38.761456 11559 init.cc:377] Logged in from keytab as oryx/127.28.154.130@KRBTEST.COM (short username oryx)
I20260811 09:08:38.763000 11559 webserver.cc:460] Webserver started at http://127.28.154.130:43103/ using document root <none> and password file <none>
I20260811 09:08:38.763689 11559 fs_manager.cc:359] Metadata directory not provided
I20260811 09:08:38.763793 11559 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:08:38.764115 11559 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:08:38.767134 11559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439250192320-29290-0/minicluster-data/ts-1/data/instance:
uuid: "1f5e4464cf2f4e57b977e56f15e323af"
format_stamp: "Formatted at 2026-08-11 09:08:38 on dist-test-slave-xt4k"
server_key: "5630bfe19f468af4e4ec542e2e7d5de3"
server_key_iv: "fd3435c4d5c332b9940ecc93344f9078"
server_key_version: "encryptionkey@0"
I20260811 09:08:38.767941 11559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439250192320-29290-0/minicluster-data/ts-1/wal/instance:
uuid: "1f5e4464cf2f4e57b977e56f15e323af"
format_stamp: "Formatted at 2026-08-11 09:08:38 on dist-test-slave-xt4k"
server_key: "5630bfe19f468af4e4ec542e2e7d5de3"
server_key_iv: "fd3435c4d5c332b9940ecc93344f9078"
server_key_version: "encryptionkey@0"
I20260811 09:08:38.773224 11559 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 09:08:38.776809 11590 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:08:38.778236 11559 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260811 09:08:38.778395 11559 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439250192320-29290-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439250192320-29290-0/minicluster-data/ts-1/wal
uuid: "1f5e4464cf2f4e57b977e56f15e323af"
format_stamp: "Formatted at 2026-08-11 09:08:38 on dist-test-slave-xt4k"
server_key: "5630bfe19f468af4e4ec542e2e7d5de3"
server_key_iv: "fd3435c4d5c332b9940ecc93344f9078"
server_key_version: "encryptionkey@0"
I20260811 09:08:38.778607 11559 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439250192320-29290-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439250192320-29290-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439250192320-29290-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:08:38.799234 11559 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:08:38.804807 11559 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:08:38.805050 11559 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:08:38.807997 11559 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:08:38.808104 11559 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:08:38.808245 11559 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:08:38.808318 11559 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 09:08:38 dist-test-slave-xt4k krb5kdc[11078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439318, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.154.130@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM
I20260811 09:08:38.828832 11338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:38.808592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.130:39421 (local address 127.28.154.190:45067)
0811 09:08:38.808789 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:38.808796 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:38.810476 (+ 1680us) server_negotiation.cc:407] Connection header received
0811 09:08:38.812835 (+ 2359us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:38.812841 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:38.812932 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:38.813027 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:38.814577 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:38.815310 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:38.817199 (+ 1889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:38.817786 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:38.817910 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:38.822807 (+ 4897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:38.822812 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:38.822816 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:08:38.822852 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:38.825343 (+ 2491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:38.826138 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:38.826145 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:38.826148 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:38.826303 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:38.826850 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:38.826853 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:38.826855 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:38.827132 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:38.827285 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:38.828133 (+ 848us) server_negotiation.cc:299] Negotiation successful
0811 09:08:38.828432 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 09:08:38.829923 11608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:38.809319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:45067 (local address 127.28.154.130:39421)
0811 09:08:38.810308 (+ 989us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:38.810348 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:38.812513 (+ 2165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:38.813279 (+ 766us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:38.813287 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:38.813812 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:08:38.814396 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:38.814415 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:38.815482 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:38.815486 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:38.816990 (+ 1504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:38.817011 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:38.818262 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:38.818270 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:38.818566 (+ 296us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:38.819440 (+ 874us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:08:38.819484 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:08:38.822581 (+ 3097us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:08:38.825519 (+ 2938us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:08:38.825529 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:08:38.825546 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:08:38.825989 (+ 443us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:08:38.826449 (+ 460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:08:38.826455 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:08:38.826460 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:08:38.826733 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:08:38.827270 (+ 537us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:08:38.827278 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:08:38.827670 (+ 392us) client_negotiation.cc:769] Sending connection context
0811 09:08:38.827974 (+ 304us) client_negotiation.cc:240] Negotiation successful
0811 09:08:38.828469 (+ 495us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":742,"thread_start_us":428,"threads_started":1}
I20260811 09:08:38.850514 11338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:38.835619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.130:45631 (local address 127.28.154.190:45067)
0811 09:08:38.835820 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:38.835827 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:38.836882 (+ 1055us) server_negotiation.cc:407] Connection header received
0811 09:08:38.837076 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:38.837082 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:38.837158 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:38.837241 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:38.838664 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:38.839569 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:38.841354 (+ 1785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:38.841857 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:38.841982 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:38.843879 (+ 1897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:38.843884 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:38.843888 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:08:38.843920 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:38.846994 (+ 3074us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:38.848087 (+ 1093us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:38.848092 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:38.848096 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:38.848257 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:38.848722 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:38.848728 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:38.848732 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:38.849256 (+ 524us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:38.849428 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:38.849923 (+ 495us) server_negotiation.cc:299] Negotiation successful
0811 09:08:38.850155 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 09:08:38.850595 11683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:38.835860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:45067 (local address 127.28.154.130:45631)
0811 09:08:38.836777 (+ 917us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:38.836793 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:38.836944 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:38.837282 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:38.837286 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:38.837909 (+ 623us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:08:38.838473 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:38.838490 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:38.839729 (+ 1239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:38.839734 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:38.841218 (+ 1484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:38.841232 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:38.842046 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:38.842053 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:38.842231 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:38.842884 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:08:38.842903 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:08:38.843668 (+ 765us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:08:38.847134 (+ 3466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:08:38.847140 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:08:38.847144 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:08:38.847931 (+ 787us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:08:38.848391 (+ 460us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:08:38.848399 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:08:38.848402 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:08:38.848586 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:08:38.849400 (+ 814us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:08:38.849405 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:08:38.849657 (+ 252us) client_negotiation.cc:769] Sending connection context
0811 09:08:38.849796 (+ 139us) client_negotiation.cc:240] Negotiation successful
0811 09:08:38.850155 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":780,"thread_start_us":443,"threads_started":1}
I20260811 09:08:38.865218 11559 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.130:34865
I20260811 09:08:38.865235 11816 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.130:34865 every 8 connection(s)
I20260811 09:08:38.866426 11559 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439250192320-29290-0/minicluster-data/ts-1/data/info.pb
I20260811 09:08:38.868744 11817 heartbeater.cc:347] Connected to a master server at 127.28.154.190:45067
I20260811 09:08:38.868955 11817 heartbeater.cc:464] Registering TS with master...
I20260811 09:08:38.869620 11817 heartbeater.cc:511] Master 127.28.154.190:45067 requested a full tablet report, sending...
I20260811 09:08:38.870920 11167 ts_manager.cc:194] Registered new tserver with Master: 1f5e4464cf2f4e57b977e56f15e323af (127.28.154.130:34865)
I20260811 09:08:38.871729 11167 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.28.154.130@KRBTEST.COM'} at 127.28.154.130:39421
I20260811 09:08:38.872421 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 11559
I20260811 09:08:38.872622 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439250192320-29290-0/minicluster-data/ts-1/wal/instance
I20260811 09:08:38.873134 29290 external_mini_cluster.cc:1428] Setting key 7c1a95cbb56ca0decec67e04045777c9
WARNING: no policy specified for oryx/127.28.154.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.28.154.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.28.154.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439250192320-29290-0/krb5kdc/oryx_127.28.154.131.keytab.
Entry for principal oryx/127.28.154.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439250192320-29290-0/krb5kdc/oryx_127.28.154.131.keytab.
I20260811 09:08:38.931824 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439250192320-29290-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439250192320-29290-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439250192320-29290-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439250192320-29290-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.154.131:0
--local_ip_for_outbound_sockets=127.28.154.131
--webserver_interface=127.28.154.131
--webserver_port=0
--tserver_master_addrs=127.28.154.190:45067
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439250192320-29290-0/krb5kdc/oryx_127.28.154.131.keytab
--principal=oryx/127.28.154.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:39899
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:08:39.275578 11824 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:08:39.275892 11824 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:08:39.276036 11824 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:08:39.283707 11824 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:08:39.283833 11824 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:08:39.283953 11824 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 09:08:39.284056 11824 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.154.131
I20260811 09:08:39.294950 11824 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:39899
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439250192320-29290-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439250192320-29290-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439250192320-29290-0/krb5kdc/oryx_127.28.154.131.keytab
--principal=oryx/127.28.154.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.154.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439250192320-29290-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.154.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.28.154.190:45067
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.154.131
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439250192320-29290-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:08:39.296453 11824 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:08:39.298202 11824 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:08:39.312278 11831 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:08:39.312325 11834 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:08:39.312397 11832 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:08:39.313452 11824 server_base.cc:1034] running on GCE node
I20260811 09:08:39.314142 11824 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:08:39.315112 11824 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:08:39.316387 11824 hybrid_clock.cc:648] HybridClock initialized: now 1786439319316279 us; error 48 us; skew 500 ppm
I20260811 09:08:39.316730 11824 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:08:39 dist-test-slave-xt4k krb5kdc[11078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439319, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.154.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:08:39.319909 11824 init.cc:377] Logged in from keytab as oryx/127.28.154.131@KRBTEST.COM (short username oryx)
I20260811 09:08:39.321532 11824 webserver.cc:460] Webserver started at http://127.28.154.131:39877/ using document root <none> and password file <none>
I20260811 09:08:39.322389 11824 fs_manager.cc:359] Metadata directory not provided
I20260811 09:08:39.322515 11824 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:08:39.322847 11824 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:08:39.326323 11824 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439250192320-29290-0/minicluster-data/ts-2/data/instance:
uuid: "b44912391db4452ba9f8e77aa97ff22b"
format_stamp: "Formatted at 2026-08-11 09:08:39 on dist-test-slave-xt4k"
server_key: "71b6d8b530fd8cfcd0a51bb14ce5094a"
server_key_iv: "ebee99787b29c2c2eba7296984d0da3e"
server_key_version: "encryptionkey@0"
I20260811 09:08:39.327483 11824 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439250192320-29290-0/minicluster-data/ts-2/wal/instance:
uuid: "b44912391db4452ba9f8e77aa97ff22b"
format_stamp: "Formatted at 2026-08-11 09:08:39 on dist-test-slave-xt4k"
server_key: "71b6d8b530fd8cfcd0a51bb14ce5094a"
server_key_iv: "ebee99787b29c2c2eba7296984d0da3e"
server_key_version: "encryptionkey@0"
I20260811 09:08:39.334307 11824 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260811 09:08:39.338057 11843 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:08:39.339490 11824 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 09:08:39.339655 11824 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439250192320-29290-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439250192320-29290-0/minicluster-data/ts-2/wal
uuid: "b44912391db4452ba9f8e77aa97ff22b"
format_stamp: "Formatted at 2026-08-11 09:08:39 on dist-test-slave-xt4k"
server_key: "71b6d8b530fd8cfcd0a51bb14ce5094a"
server_key_iv: "ebee99787b29c2c2eba7296984d0da3e"
server_key_version: "encryptionkey@0"
I20260811 09:08:39.339991 11548 heartbeater.cc:503] Master 127.28.154.190:45067 was elected leader, sending a full tablet report...
I20260811 09:08:39.339919 11824 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439250192320-29290-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439250192320-29290-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439250192320-29290-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:08:39.355230 11824 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:08:39.359959 11824 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:08:39.360370 11824 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:08:39.363556 11824 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:08:39.363677 11824 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:08:39.363821 11824 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:08:39.363905 11824 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 09:08:39 dist-test-slave-xt4k krb5kdc[11078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439319, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.154.131@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM
I20260811 09:08:39.388906 11852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:39.363968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.131:41529 (local address 127.28.154.190:45067)
0811 09:08:39.364479 (+ 511us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:39.364486 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:39.366020 (+ 1534us) server_negotiation.cc:407] Connection header received
0811 09:08:39.368489 (+ 2469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:39.368495 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:39.368620 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:39.368721 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:39.371047 (+ 2326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:39.372065 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:39.375103 (+ 3038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:39.375652 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:39.375805 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:39.380957 (+ 5152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:39.380968 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:39.380973 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:08:39.381031 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:39.385015 (+ 3984us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:39.385901 (+ 886us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:39.385908 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:39.385913 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:39.386106 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:39.386693 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:39.386793 (+ 100us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:39.386796 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:39.387193 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:39.387357 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:39.388219 (+ 862us) server_negotiation.cc:299] Negotiation successful
0811 09:08:39.388510 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":378,"thread_start_us":228,"threads_started":1}
I20260811 09:08:39.390038 11856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:39.364793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:45067 (local address 127.28.154.131:41529)
0811 09:08:39.365825 (+ 1032us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:39.365867 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:39.368149 (+ 2282us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:39.369030 (+ 881us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:39.369044 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:39.369911 (+ 867us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:08:39.370838 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:39.370870 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:39.372262 (+ 1392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:39.372272 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:39.374921 (+ 2649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:39.374939 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:39.375949 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:39.375963 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:39.376299 (+ 336us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:39.377294 (+ 995us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:08:39.377319 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:08:39.380599 (+ 3280us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:08:39.385186 (+ 4587us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:08:39.385195 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:08:39.385212 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:08:39.385778 (+ 566us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:08:39.386243 (+ 465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:08:39.386249 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:08:39.386253 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:08:39.386558 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:08:39.387335 (+ 777us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:08:39.387344 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:08:39.387817 (+ 473us) client_negotiation.cc:769] Sending connection context
0811 09:08:39.388091 (+ 274us) client_negotiation.cc:240] Negotiation successful
0811 09:08:39.388726 (+ 635us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":752,"thread_start_us":502,"threads_started":1}
I20260811 09:08:39.411588 11852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:39.396269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.131:49189 (local address 127.28.154.190:45067)
0811 09:08:39.396469 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:39.396476 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:39.397273 (+ 797us) server_negotiation.cc:407] Connection header received
0811 09:08:39.397521 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:39.397527 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:39.397628 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:39.397703 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:39.399192 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:39.400139 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:39.402217 (+ 2078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:39.402815 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:39.402933 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:39.405476 (+ 2543us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:39.405482 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:39.405485 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:08:39.405525 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:39.408874 (+ 3349us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:39.409583 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:39.409588 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:39.409590 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:39.409727 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:39.410097 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:39.410102 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:39.410106 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:39.410394 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:39.410586 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:39.410976 (+ 390us) server_negotiation.cc:299] Negotiation successful
0811 09:08:39.411198 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 09:08:39.411669 11986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:39.396398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:45067 (local address 127.28.154.131:49189)
0811 09:08:39.397134 (+ 736us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:39.397159 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:39.397329 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:39.397930 (+ 601us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:39.397935 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:39.398556 (+ 621us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:08:39.399028 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:39.399044 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:39.400411 (+ 1367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:39.400417 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:39.402066 (+ 1649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:39.402087 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:39.403013 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:39.403020 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:39.403195 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:39.403946 (+ 751us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:08:39.403965 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:08:39.405257 (+ 1292us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:08:39.409026 (+ 3769us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:08:39.409032 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:08:39.409037 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:08:39.409437 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:08:39.409835 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:08:39.409841 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:08:39.409845 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:08:39.409974 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:08:39.410511 (+ 537us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:08:39.410516 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:08:39.410708 (+ 192us) client_negotiation.cc:769] Sending connection context
0811 09:08:39.410838 (+ 130us) client_negotiation.cc:240] Negotiation successful
0811 09:08:39.411223 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":548,"thread_start_us":388,"threads_started":1}
I20260811 09:08:39.417285 11824 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.131:39043
I20260811 09:08:39.417307 12053 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.131:39043 every 8 connection(s)
I20260811 09:08:39.418499 11824 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439250192320-29290-0/minicluster-data/ts-2/data/info.pb
I20260811 09:08:39.420569 12058 heartbeater.cc:347] Connected to a master server at 127.28.154.190:45067
I20260811 09:08:39.420799 12058 heartbeater.cc:464] Registering TS with master...
I20260811 09:08:39.421453 12058 heartbeater.cc:511] Master 127.28.154.190:45067 requested a full tablet report, sending...
I20260811 09:08:39.422833 11167 ts_manager.cc:194] Registered new tserver with Master: b44912391db4452ba9f8e77aa97ff22b (127.28.154.131:39043)
I20260811 09:08:39.423640 11167 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.28.154.131@KRBTEST.COM'} at 127.28.154.131:41529
I20260811 09:08:39.428736 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 11824
I20260811 09:08:39.428916 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439250192320-29290-0/minicluster-data/ts-2/wal/instance
I20260811 09:08:39.429320 29290 external_mini_cluster.cc:1428] Setting key 5b9cf29f1ad7a6d6fa8f319b66cf2360
I20260811 09:08:39.432897 29290 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 09:08:39 dist-test-slave-xt4k krb5kdc[11078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439319, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 09:08:39 dist-test-slave-xt4k krb5kdc[11078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.28.154.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:08:39 dist-test-slave-xt4k krb5kdc[11078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.28.154.190@KRBTEST.COM, Server not found in Kerberos database
W20260811 09:08:39.496170 11852 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:08:39.485599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55708 (local address 127.28.154.190:45067)
0811 09:08:39.485779 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:39.485784 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:39.486000 (+ 216us) server_negotiation.cc:407] Connection header received
0811 09:08:39.486172 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:39.486176 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:39.486244 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:39.486337 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:39.487636 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:39.488385 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:39.489665 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:39.490074 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:39.490180 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:39.495930 (+ 5750us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55708: BlockingRecv error: recv got EOF from 127.0.0.1:55708 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 11 09:08:39 dist-test-slave-xt4k krb5kdc[11078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439319, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM
I20260811 09:08:39.512328 11852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:39.499087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55718 (local address 127.28.154.190:45067)
0811 09:08:39.499289 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:39.499296 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:39.499554 (+ 258us) server_negotiation.cc:407] Connection header received
0811 09:08:39.499678 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:39.499682 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:39.499773 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:39.499860 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:39.501178 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:39.501903 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:39.503092 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:39.503505 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:39.503605 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:39.507217 (+ 3612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:39.507224 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:39.507227 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:08:39.507265 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:39.509808 (+ 2543us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:39.510509 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:39.510513 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:39.510538 (+ 25us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:39.510667 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:39.511062 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:39.511065 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:39.511067 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:39.511343 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:39.511545 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:39.511858 (+ 313us) server_negotiation.cc:299] Negotiation successful
0811 09:08:39.512036 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 09:08:39.516422 11167 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55718:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 09:08:39.523337 11167 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 09:08:39.557260 12107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:39.536610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40632 (local address 127.28.154.131:39043)
0811 09:08:39.548546 (+ 11936us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:39.548555 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:39.549062 (+ 507us) server_negotiation.cc:407] Connection header received
0811 09:08:39.549162 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:39.549170 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:39.549370 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:39.550342 (+ 972us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:08:39.550664 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:39.552159 (+ 1495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:39.554783 (+ 2624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:39.556017 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:39.556207 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:39.556424 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:39.556746 (+ 322us) server_negotiation.cc:299] Negotiation successful
0811 09:08:39.556907 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11705,"thread_start_us":11506,"threads_started":1}
I20260811 09:08:39.557410 12103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:39.536218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42752 (local address 127.28.154.130:34865)
0811 09:08:39.548733 (+ 12515us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:39.548742 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:39.548775 (+ 33us) server_negotiation.cc:407] Connection header received
0811 09:08:39.548893 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:39.548901 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:39.549181 (+ 280us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:39.549485 (+ 304us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:08:39.550344 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:39.551829 (+ 1485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:39.555104 (+ 3275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:39.556209 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:39.556402 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:39.556573 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:39.556946 (+ 373us) server_negotiation.cc:299] Negotiation successful
0811 09:08:39.557110 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12338,"thread_start_us":12360,"threads_started":1}
I20260811 09:08:39.557705 12099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:39.536173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.131:39043 (local address 127.0.0.1:40632)
0811 09:08:39.548795 (+ 12622us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:39.548814 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:39.548943 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:39.549625 (+ 682us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:39.549632 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:39.549660 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:08:39.550112 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:39.550129 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:39.552308 (+ 2179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:39.552313 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:39.554588 (+ 2275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:39.554604 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:39.556234 (+ 1630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:39.556241 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:39.556482 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:39.556540 (+ 58us) client_negotiation.cc:769] Sending connection context
0811 09:08:39.556677 (+ 137us) client_negotiation.cc:240] Negotiation successful
0811 09:08:39.556875 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12490,"mutex_wait_us":40,"thread_start_us":9860,"threads_started":1}
I20260811 09:08:39.558434 12105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:39.536061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.130:34865 (local address 127.0.0.1:42752)
0811 09:08:39.546527 (+ 10466us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:39.546583 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:39.546818 (+ 235us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:39.549714 (+ 2896us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:39.549718 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:39.549742 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:08:39.550166 (+ 424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:39.550175 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:39.552008 (+ 1833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:39.552013 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:39.554963 (+ 2950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:39.554976 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:39.556583 (+ 1607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:39.556590 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:39.556753 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:39.556786 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 09:08:39.557879 (+ 1093us) client_negotiation.cc:240] Negotiation successful
0811 09:08:39.558021 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10131,"mutex_wait_us":459,"thread_start_us":12473,"threads_started":1}
I20260811 09:08:39.561631 11935 tablet_service.cc:1467] Processing CreateTablet for tablet c1d8449979344cdcbc63b1a0961ae989 (DEFAULT_TABLE table=test-table [id=e2d76311f51a4ab094579348b31d3b99]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:08:39.561852 11697 tablet_service.cc:1467] Processing CreateTablet for tablet c1d8449979344cdcbc63b1a0961ae989 (DEFAULT_TABLE table=test-table [id=e2d76311f51a4ab094579348b31d3b99]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:08:39.562559 12110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:39.548980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.129:41463 (local address 127.0.0.1:41522)
0811 09:08:39.554082 (+ 5102us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:39.554099 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:39.554219 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:39.555039 (+ 820us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:39.555043 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:39.555067 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:08:39.555438 (+ 371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:39.555461 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:39.557249 (+ 1788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:39.557255 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:39.559560 (+ 2305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:39.559575 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:39.561717 (+ 2142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:39.561724 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:39.561898 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:39.561932 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 09:08:39.562040 (+ 108us) client_negotiation.cc:240] Negotiation successful
0811 09:08:39.562169 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4983,"thread_start_us":4805,"threads_started":1}
I20260811 09:08:39.563222 11935 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c1d8449979344cdcbc63b1a0961ae989. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:08:39.563218 12112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:39.549232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41522 (local address 127.28.154.129:41463)
0811 09:08:39.553676 (+ 4444us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:39.553685 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:39.554338 (+ 653us) server_negotiation.cc:407] Connection header received
0811 09:08:39.554436 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:39.554443 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:39.554676 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:39.554850 (+ 174us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:08:39.555643 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:39.557068 (+ 1425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:39.559749 (+ 2681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:39.561184 (+ 1435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:39.561393 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:39.561560 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:39.562766 (+ 1206us) server_negotiation.cc:299] Negotiation successful
0811 09:08:39.562948 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4281,"thread_start_us":4148,"threads_started":1}
I20260811 09:08:39.563246 11697 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c1d8449979344cdcbc63b1a0961ae989. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:08:39.566747 11410 tablet_service.cc:1467] Processing CreateTablet for tablet c1d8449979344cdcbc63b1a0961ae989 (DEFAULT_TABLE table=test-table [id=e2d76311f51a4ab094579348b31d3b99]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:08:39.568181 11410 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c1d8449979344cdcbc63b1a0961ae989. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:08:39.575474 12113 tablet_bootstrap.cc:492] T c1d8449979344cdcbc63b1a0961ae989 P b44912391db4452ba9f8e77aa97ff22b: Bootstrap starting.
I20260811 09:08:39.577519 12114 tablet_bootstrap.cc:492] T c1d8449979344cdcbc63b1a0961ae989 P 1f5e4464cf2f4e57b977e56f15e323af: Bootstrap starting.
I20260811 09:08:39.580183 12114 tablet_bootstrap.cc:654] T c1d8449979344cdcbc63b1a0961ae989 P 1f5e4464cf2f4e57b977e56f15e323af: Neither blocks nor log segments found. Creating new log.
I20260811 09:08:39.580221 12113 tablet_bootstrap.cc:654] T c1d8449979344cdcbc63b1a0961ae989 P b44912391db4452ba9f8e77aa97ff22b: Neither blocks nor log segments found. Creating new log.
I20260811 09:08:39.581194 12114 log.cc:824] T c1d8449979344cdcbc63b1a0961ae989 P 1f5e4464cf2f4e57b977e56f15e323af: Log is configured to *not* fsync() on all Append() calls
I20260811 09:08:39.581425 12113 log.cc:824] T c1d8449979344cdcbc63b1a0961ae989 P b44912391db4452ba9f8e77aa97ff22b: Log is configured to *not* fsync() on all Append() calls
I20260811 09:08:39.583236 12119 tablet_bootstrap.cc:492] T c1d8449979344cdcbc63b1a0961ae989 P e6ba0c4707a9480b8309f094af83bd48: Bootstrap starting.
I20260811 09:08:39.584465 12114 tablet_bootstrap.cc:492] T c1d8449979344cdcbc63b1a0961ae989 P 1f5e4464cf2f4e57b977e56f15e323af: No bootstrap required, opened a new log
I20260811 09:08:39.584750 12114 ts_tablet_manager.cc:1397] T c1d8449979344cdcbc63b1a0961ae989 P 1f5e4464cf2f4e57b977e56f15e323af: Time spent bootstrapping tablet: real 0.007s user 0.007s sys 0.000s
I20260811 09:08:39.586545 12113 tablet_bootstrap.cc:492] T c1d8449979344cdcbc63b1a0961ae989 P b44912391db4452ba9f8e77aa97ff22b: No bootstrap required, opened a new log
I20260811 09:08:39.586942 12113 ts_tablet_manager.cc:1397] T c1d8449979344cdcbc63b1a0961ae989 P b44912391db4452ba9f8e77aa97ff22b: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.000s
I20260811 09:08:39.587527 12119 tablet_bootstrap.cc:654] T c1d8449979344cdcbc63b1a0961ae989 P e6ba0c4707a9480b8309f094af83bd48: Neither blocks nor log segments found. Creating new log.
I20260811 09:08:39.588835 12119 log.cc:824] T c1d8449979344cdcbc63b1a0961ae989 P e6ba0c4707a9480b8309f094af83bd48: Log is configured to *not* fsync() on all Append() calls
I20260811 09:08:39.592134 12119 tablet_bootstrap.cc:492] T c1d8449979344cdcbc63b1a0961ae989 P e6ba0c4707a9480b8309f094af83bd48: No bootstrap required, opened a new log
I20260811 09:08:39.592428 12119 ts_tablet_manager.cc:1397] T c1d8449979344cdcbc63b1a0961ae989 P e6ba0c4707a9480b8309f094af83bd48: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.003s
I20260811 09:08:39.594864 12114 raft_consensus.cc:348] T c1d8449979344cdcbc63b1a0961ae989 P 1f5e4464cf2f4e57b977e56f15e323af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f5e4464cf2f4e57b977e56f15e323af" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34865 } } peers { permanent_uuid: "b44912391db4452ba9f8e77aa97ff22b" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 39043 } } peers { permanent_uuid: "e6ba0c4707a9480b8309f094af83bd48" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41463 } }
I20260811 09:08:39.594992 12113 raft_consensus.cc:348] T c1d8449979344cdcbc63b1a0961ae989 P b44912391db4452ba9f8e77aa97ff22b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f5e4464cf2f4e57b977e56f15e323af" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34865 } } peers { permanent_uuid: "b44912391db4452ba9f8e77aa97ff22b" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 39043 } } peers { permanent_uuid: "e6ba0c4707a9480b8309f094af83bd48" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41463 } }
I20260811 09:08:39.595340 12113 raft_consensus.cc:374] T c1d8449979344cdcbc63b1a0961ae989 P b44912391db4452ba9f8e77aa97ff22b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:08:39.595341 12114 raft_consensus.cc:374] T c1d8449979344cdcbc63b1a0961ae989 P 1f5e4464cf2f4e57b977e56f15e323af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:08:39.595438 12113 raft_consensus.cc:729] T c1d8449979344cdcbc63b1a0961ae989 P b44912391db4452ba9f8e77aa97ff22b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b44912391db4452ba9f8e77aa97ff22b, State: Initialized, Role: FOLLOWER
I20260811 09:08:39.595432 12114 raft_consensus.cc:729] T c1d8449979344cdcbc63b1a0961ae989 P 1f5e4464cf2f4e57b977e56f15e323af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f5e4464cf2f4e57b977e56f15e323af, State: Initialized, Role: FOLLOWER
I20260811 09:08:39.596091 12113 consensus_queue.cc:260] T c1d8449979344cdcbc63b1a0961ae989 P b44912391db4452ba9f8e77aa97ff22b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f5e4464cf2f4e57b977e56f15e323af" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34865 } } peers { permanent_uuid: "b44912391db4452ba9f8e77aa97ff22b" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 39043 } } peers { permanent_uuid: "e6ba0c4707a9480b8309f094af83bd48" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41463 } }
I20260811 09:08:39.596114 12114 consensus_queue.cc:260] T c1d8449979344cdcbc63b1a0961ae989 P 1f5e4464cf2f4e57b977e56f15e323af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f5e4464cf2f4e57b977e56f15e323af" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34865 } } peers { permanent_uuid: "b44912391db4452ba9f8e77aa97ff22b" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 39043 } } peers { permanent_uuid: "e6ba0c4707a9480b8309f094af83bd48" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41463 } }
I20260811 09:08:39.597384 12058 heartbeater.cc:503] Master 127.28.154.190:45067 was elected leader, sending a full tablet report...
I20260811 09:08:39.597829 12113 ts_tablet_manager.cc:1428] T c1d8449979344cdcbc63b1a0961ae989 P b44912391db4452ba9f8e77aa97ff22b: Time spent starting tablet: real 0.011s user 0.011s sys 0.000s
I20260811 09:08:39.601186 11817 heartbeater.cc:503] Master 127.28.154.190:45067 was elected leader, sending a full tablet report...
I20260811 09:08:39.602667 12114 ts_tablet_manager.cc:1428] T c1d8449979344cdcbc63b1a0961ae989 P 1f5e4464cf2f4e57b977e56f15e323af: Time spent starting tablet: real 0.018s user 0.008s sys 0.008s
I20260811 09:08:39.603853 12119 raft_consensus.cc:348] T c1d8449979344cdcbc63b1a0961ae989 P e6ba0c4707a9480b8309f094af83bd48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f5e4464cf2f4e57b977e56f15e323af" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34865 } } peers { permanent_uuid: "b44912391db4452ba9f8e77aa97ff22b" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 39043 } } peers { permanent_uuid: "e6ba0c4707a9480b8309f094af83bd48" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41463 } }
I20260811 09:08:39.604321 12119 raft_consensus.cc:374] T c1d8449979344cdcbc63b1a0961ae989 P e6ba0c4707a9480b8309f094af83bd48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:08:39.604463 12119 raft_consensus.cc:729] T c1d8449979344cdcbc63b1a0961ae989 P e6ba0c4707a9480b8309f094af83bd48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6ba0c4707a9480b8309f094af83bd48, State: Initialized, Role: FOLLOWER
I20260811 09:08:39.605074 12119 consensus_queue.cc:260] T c1d8449979344cdcbc63b1a0961ae989 P e6ba0c4707a9480b8309f094af83bd48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f5e4464cf2f4e57b977e56f15e323af" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34865 } } peers { permanent_uuid: "b44912391db4452ba9f8e77aa97ff22b" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 39043 } } peers { permanent_uuid: "e6ba0c4707a9480b8309f094af83bd48" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41463 } }
I20260811 09:08:39.607179 12119 ts_tablet_manager.cc:1428] T c1d8449979344cdcbc63b1a0961ae989 P e6ba0c4707a9480b8309f094af83bd48: Time spent starting tablet: real 0.015s user 0.008s sys 0.004s
W20260811 09:08:39.618480 11818 tablet.cc:2369] T c1d8449979344cdcbc63b1a0961ae989 P 1f5e4464cf2f4e57b977e56f15e323af: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 09:08:39.670213 12059 tablet.cc:2369] T c1d8449979344cdcbc63b1a0961ae989 P b44912391db4452ba9f8e77aa97ff22b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 09:08:39.833742 11550 tablet.cc:2369] T c1d8449979344cdcbc63b1a0961ae989 P e6ba0c4707a9480b8309f094af83bd48: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 09:08:39.946973 12124 raft_consensus.cc:482] T c1d8449979344cdcbc63b1a0961ae989 P e6ba0c4707a9480b8309f094af83bd48 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:08:39.947325 12124 raft_consensus.cc:504] T c1d8449979344cdcbc63b1a0961ae989 P e6ba0c4707a9480b8309f094af83bd48 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f5e4464cf2f4e57b977e56f15e323af" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34865 } } peers { permanent_uuid: "b44912391db4452ba9f8e77aa97ff22b" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 39043 } } peers { permanent_uuid: "e6ba0c4707a9480b8309f094af83bd48" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41463 } }
I20260811 09:08:39.949121 12124 leader_election.cc:288] T c1d8449979344cdcbc63b1a0961ae989 P e6ba0c4707a9480b8309f094af83bd48 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f5e4464cf2f4e57b977e56f15e323af (127.28.154.130:34865), b44912391db4452ba9f8e77aa97ff22b (127.28.154.131:39043)
I20260811 09:08:39.958231 12107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:39.949679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.129:43973 (local address 127.28.154.131:39043)
0811 09:08:39.949859 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:39.949865 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:39.950640 (+ 775us) server_negotiation.cc:407] Connection header received
0811 09:08:39.950801 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:39.950807 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:39.950878 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:39.950965 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:08:39.951689 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:39.952885 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:39.955810 (+ 2925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:39.957072 (+ 1262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:39.957235 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:39.957314 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:39.957822 (+ 508us) server_negotiation.cc:299] Negotiation successful
0811 09:08:39.957959 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 09:08:39.958240 12103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:39.949671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.129:36633 (local address 127.28.154.130:34865)
0811 09:08:39.949861 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:39.949866 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:39.950581 (+ 715us) server_negotiation.cc:407] Connection header received
0811 09:08:39.950761 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:39.950767 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:39.950855 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:39.950954 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:08:39.951712 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:39.952885 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:39.955810 (+ 2925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:39.957026 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:39.957235 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:39.957314 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:39.957821 (+ 507us) server_negotiation.cc:299] Negotiation successful
0811 09:08:39.957959 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 09:08:39.958441 12127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:39.949519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.131:39043 (local address 127.28.154.129:43973)
0811 09:08:39.950435 (+ 916us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:39.950455 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:39.950652 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:39.951002 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:39.951007 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:39.951043 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:08:39.951530 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:39.951547 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:39.953032 (+ 1485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:39.953038 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:39.955606 (+ 2568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:39.955624 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:39.957319 (+ 1695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:39.957325 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:39.957616 (+ 291us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:39.957693 (+ 77us) client_negotiation.cc:769] Sending connection context
0811 09:08:39.957790 (+ 97us) client_negotiation.cc:240] Negotiation successful
0811 09:08:39.957982 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":740,"mutex_wait_us":211,"spinlock_wait_cycles":23808,"thread_start_us":509,"threads_started":1}
I20260811 09:08:39.959070 12129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:39.949609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.130:34865 (local address 127.28.154.129:36633)
0811 09:08:39.950435 (+ 826us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:39.950455 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:39.950622 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:39.951101 (+ 479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:39.951106 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:39.951173 (+ 67us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:08:39.951560 (+ 387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:39.951570 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:39.953038 (+ 1468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:39.953042 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:39.955606 (+ 2564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:39.955624 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:39.957296 (+ 1672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:39.957302 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:39.957614 (+ 312us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:39.957683 (+ 69us) client_negotiation.cc:769] Sending connection context
0811 09:08:39.957783 (+ 100us) client_negotiation.cc:240] Negotiation successful
0811 09:08:39.957982 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":687,"mutex_wait_us":239,"spinlock_wait_cycles":27904,"thread_start_us":436,"threads_started":1}
I20260811 09:08:39.959430 11973 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1d8449979344cdcbc63b1a0961ae989" candidate_uuid: "e6ba0c4707a9480b8309f094af83bd48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b44912391db4452ba9f8e77aa97ff22b" is_pre_election: true
I20260811 09:08:39.959873 11973 raft_consensus.cc:2454] T c1d8449979344cdcbc63b1a0961ae989 P b44912391db4452ba9f8e77aa97ff22b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6ba0c4707a9480b8309f094af83bd48 in term 0.
I20260811 09:08:39.959957 11737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1d8449979344cdcbc63b1a0961ae989" candidate_uuid: "e6ba0c4707a9480b8309f094af83bd48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f5e4464cf2f4e57b977e56f15e323af" is_pre_election: true
I20260811 09:08:39.960443 11737 raft_consensus.cc:2454] T c1d8449979344cdcbc63b1a0961ae989 P 1f5e4464cf2f4e57b977e56f15e323af [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6ba0c4707a9480b8309f094af83bd48 in term 0.
I20260811 09:08:39.960672 11325 leader_election.cc:302] T c1d8449979344cdcbc63b1a0961ae989 P e6ba0c4707a9480b8309f094af83bd48 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b44912391db4452ba9f8e77aa97ff22b, e6ba0c4707a9480b8309f094af83bd48; no voters:
I20260811 09:08:39.961055 12124 raft_consensus.cc:2781] T c1d8449979344cdcbc63b1a0961ae989 P e6ba0c4707a9480b8309f094af83bd48 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 09:08:39.961184 12124 raft_consensus.cc:482] T c1d8449979344cdcbc63b1a0961ae989 P e6ba0c4707a9480b8309f094af83bd48 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 09:08:39.961279 12124 raft_consensus.cc:3037] T c1d8449979344cdcbc63b1a0961ae989 P e6ba0c4707a9480b8309f094af83bd48 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:08:39.963450 12124 raft_consensus.cc:504] T c1d8449979344cdcbc63b1a0961ae989 P e6ba0c4707a9480b8309f094af83bd48 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f5e4464cf2f4e57b977e56f15e323af" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34865 } } peers { permanent_uuid: "b44912391db4452ba9f8e77aa97ff22b" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 39043 } } peers { permanent_uuid: "e6ba0c4707a9480b8309f094af83bd48" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41463 } }
I20260811 09:08:39.964066 12124 leader_election.cc:288] T c1d8449979344cdcbc63b1a0961ae989 P e6ba0c4707a9480b8309f094af83bd48 [CANDIDATE]: Term 1 election: Requested vote from peers 1f5e4464cf2f4e57b977e56f15e323af (127.28.154.130:34865), b44912391db4452ba9f8e77aa97ff22b (127.28.154.131:39043)
I20260811 09:08:39.964507 11737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1d8449979344cdcbc63b1a0961ae989" candidate_uuid: "e6ba0c4707a9480b8309f094af83bd48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f5e4464cf2f4e57b977e56f15e323af"
I20260811 09:08:39.964529 11973 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1d8449979344cdcbc63b1a0961ae989" candidate_uuid: "e6ba0c4707a9480b8309f094af83bd48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b44912391db4452ba9f8e77aa97ff22b"
I20260811 09:08:39.964694 11737 raft_consensus.cc:3037] T c1d8449979344cdcbc63b1a0961ae989 P 1f5e4464cf2f4e57b977e56f15e323af [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:08:39.964706 11973 raft_consensus.cc:3037] T c1d8449979344cdcbc63b1a0961ae989 P b44912391db4452ba9f8e77aa97ff22b [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:08:39.966724 11973 raft_consensus.cc:2454] T c1d8449979344cdcbc63b1a0961ae989 P b44912391db4452ba9f8e77aa97ff22b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6ba0c4707a9480b8309f094af83bd48 in term 1.
I20260811 09:08:39.966784 11737 raft_consensus.cc:2454] T c1d8449979344cdcbc63b1a0961ae989 P 1f5e4464cf2f4e57b977e56f15e323af [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6ba0c4707a9480b8309f094af83bd48 in term 1.
I20260811 09:08:39.967077 11325 leader_election.cc:302] T c1d8449979344cdcbc63b1a0961ae989 P e6ba0c4707a9480b8309f094af83bd48 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b44912391db4452ba9f8e77aa97ff22b, e6ba0c4707a9480b8309f094af83bd48; no voters:
I20260811 09:08:39.967347 12124 raft_consensus.cc:2781] T c1d8449979344cdcbc63b1a0961ae989 P e6ba0c4707a9480b8309f094af83bd48 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:08:39.967730 12124 raft_consensus.cc:686] T c1d8449979344cdcbc63b1a0961ae989 P e6ba0c4707a9480b8309f094af83bd48 [term 1 LEADER]: Becoming Leader. State: Replica: e6ba0c4707a9480b8309f094af83bd48, State: Running, Role: LEADER
I20260811 09:08:39.968173 12124 consensus_queue.cc:237] T c1d8449979344cdcbc63b1a0961ae989 P e6ba0c4707a9480b8309f094af83bd48 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f5e4464cf2f4e57b977e56f15e323af" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34865 } } peers { permanent_uuid: "b44912391db4452ba9f8e77aa97ff22b" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 39043 } } peers { permanent_uuid: "e6ba0c4707a9480b8309f094af83bd48" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41463 } }
I20260811 09:08:39.972764 11162 catalog_manager.cc:5495] T c1d8449979344cdcbc63b1a0961ae989 P e6ba0c4707a9480b8309f094af83bd48 reported cstate change: term changed from 0 to 1, leader changed from <none> to e6ba0c4707a9480b8309f094af83bd48 (127.28.154.129). New cstate: current_term: 1 leader_uuid: "e6ba0c4707a9480b8309f094af83bd48" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f5e4464cf2f4e57b977e56f15e323af" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34865 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b44912391db4452ba9f8e77aa97ff22b" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 39043 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e6ba0c4707a9480b8309f094af83bd48" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41463 } health_report { overall_health: HEALTHY } } }
I20260811 09:08:40.022883 12112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:40.017011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41528 (local address 127.28.154.129:41463)
0811 09:08:40.017269 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:40.017276 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:40.017652 (+ 376us) server_negotiation.cc:407] Connection header received
0811 09:08:40.017888 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:40.017892 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:40.017997 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:40.018138 (+ 141us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:08:40.018587 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:40.019304 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:40.020719 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:40.021148 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:40.021264 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:40.021676 (+ 412us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:08:40.022242 (+ 566us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:08:40.022398 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:40.022520 (+ 122us) server_negotiation.cc:299] Negotiation successful
0811 09:08:40.022608 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260811 09:08:40.031533 11737 raft_consensus.cc:1261] T c1d8449979344cdcbc63b1a0961ae989 P 1f5e4464cf2f4e57b977e56f15e323af [term 1 FOLLOWER]: Refusing update from remote peer e6ba0c4707a9480b8309f094af83bd48: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:08:40.031533 11973 raft_consensus.cc:1261] T c1d8449979344cdcbc63b1a0961ae989 P b44912391db4452ba9f8e77aa97ff22b [term 1 FOLLOWER]: Refusing update from remote peer e6ba0c4707a9480b8309f094af83bd48: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:08:40.032954 12124 consensus_queue.cc:1035] T c1d8449979344cdcbc63b1a0961ae989 P e6ba0c4707a9480b8309f094af83bd48 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f5e4464cf2f4e57b977e56f15e323af" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34865 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:08:40.033320 12131 consensus_queue.cc:1035] T c1d8449979344cdcbc63b1a0961ae989 P e6ba0c4707a9480b8309f094af83bd48 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b44912391db4452ba9f8e77aa97ff22b" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 39043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:08:40.061705 11167 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55718:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:08:40.062021 11167 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55718:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:08:40.066020 11167 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 32c27303c7e1421abd79752385c82cd5: Sending DeleteTablet for 3 replicas of tablet c1d8449979344cdcbc63b1a0961ae989
I20260811 09:08:40.067368 11935 tablet_service.cc:1514] Processing DeleteTablet for tablet c1d8449979344cdcbc63b1a0961ae989 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:08:40 UTC) from {username='oryx', principal='oryx/127.28.154.190@KRBTEST.COM'} at 127.0.0.1:40632
I20260811 09:08:40.067628 11697 tablet_service.cc:1514] Processing DeleteTablet for tablet c1d8449979344cdcbc63b1a0961ae989 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:08:40 UTC) from {username='oryx', principal='oryx/127.28.154.190@KRBTEST.COM'} at 127.0.0.1:42752
I20260811 09:08:40.068728 11167 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55718:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 09:08:40.068985 11410 tablet_service.cc:1514] Processing DeleteTablet for tablet c1d8449979344cdcbc63b1a0961ae989 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:08:40 UTC) from {username='oryx', principal='oryx/127.28.154.190@KRBTEST.COM'} at 127.0.0.1:41522
I20260811 09:08:40.069164 12159 tablet_replica.cc:330] T c1d8449979344cdcbc63b1a0961ae989 P 1f5e4464cf2f4e57b977e56f15e323af: stopping tablet replica
W20260811 09:08:40.069413 11167 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 09:08:40.069617 12159 raft_consensus.cc:2229] T c1d8449979344cdcbc63b1a0961ae989 P 1f5e4464cf2f4e57b977e56f15e323af [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:08:40.070191 12159 raft_consensus.cc:2258] T c1d8449979344cdcbc63b1a0961ae989 P 1f5e4464cf2f4e57b977e56f15e323af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:08:40.070740 12158 tablet_replica.cc:330] T c1d8449979344cdcbc63b1a0961ae989 P b44912391db4452ba9f8e77aa97ff22b: stopping tablet replica
I20260811 09:08:40.071178 12158 raft_consensus.cc:2229] T c1d8449979344cdcbc63b1a0961ae989 P b44912391db4452ba9f8e77aa97ff22b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:08:40.071683 12158 raft_consensus.cc:2258] T c1d8449979344cdcbc63b1a0961ae989 P b44912391db4452ba9f8e77aa97ff22b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:08:40.073228 12159 ts_tablet_manager.cc:1905] T c1d8449979344cdcbc63b1a0961ae989 P 1f5e4464cf2f4e57b977e56f15e323af: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:08:40.073668 12158 ts_tablet_manager.cc:1905] T c1d8449979344cdcbc63b1a0961ae989 P b44912391db4452ba9f8e77aa97ff22b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:08:40.077239 12160 tablet_replica.cc:330] T c1d8449979344cdcbc63b1a0961ae989 P e6ba0c4707a9480b8309f094af83bd48: stopping tablet replica
I20260811 09:08:40.080834 12159 ts_tablet_manager.cc:1918] T c1d8449979344cdcbc63b1a0961ae989 P 1f5e4464cf2f4e57b977e56f15e323af: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:08:40.081058 12159 log.cc:1196] T c1d8449979344cdcbc63b1a0961ae989 P 1f5e4464cf2f4e57b977e56f15e323af: Deleting WAL directory at /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439250192320-29290-0/minicluster-data/ts-1/wal/wals/c1d8449979344cdcbc63b1a0961ae989
I20260811 09:08:40.081003 11935 tablet_service.cc:1467] Processing CreateTablet for tablet f6ef2c2424dd4e49af7db09c1114908d (DEFAULT_TABLE table=test-table [id=26aa0a54b73347ff9b5658ca0c391075]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:08:40.081575 11935 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f6ef2c2424dd4e49af7db09c1114908d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:08:40.081693 12159 ts_tablet_manager.cc:1939] T c1d8449979344cdcbc63b1a0961ae989 P 1f5e4464cf2f4e57b977e56f15e323af: Deleting consensus metadata
I20260811 09:08:40.081750 12158 ts_tablet_manager.cc:1918] T c1d8449979344cdcbc63b1a0961ae989 P b44912391db4452ba9f8e77aa97ff22b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:08:40.081928 12158 log.cc:1196] T c1d8449979344cdcbc63b1a0961ae989 P b44912391db4452ba9f8e77aa97ff22b: Deleting WAL directory at /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439250192320-29290-0/minicluster-data/ts-2/wal/wals/c1d8449979344cdcbc63b1a0961ae989
I20260811 09:08:40.082476 12158 ts_tablet_manager.cc:1939] T c1d8449979344cdcbc63b1a0961ae989 P b44912391db4452ba9f8e77aa97ff22b: Deleting consensus metadata
I20260811 09:08:40.077713 12160 raft_consensus.cc:2229] T c1d8449979344cdcbc63b1a0961ae989 P e6ba0c4707a9480b8309f094af83bd48 [term 1 LEADER]: Raft consensus shutting down.
I20260811 09:08:40.084641 11142 catalog_manager.cc:4841] TS b44912391db4452ba9f8e77aa97ff22b (127.28.154.131:39043): tablet c1d8449979344cdcbc63b1a0961ae989 (table test-table [id=e2d76311f51a4ab094579348b31d3b99]) successfully deleted
I20260811 09:08:40.084854 12160 raft_consensus.cc:2258] T c1d8449979344cdcbc63b1a0961ae989 P e6ba0c4707a9480b8309f094af83bd48 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:08:40.086010 11141 catalog_manager.cc:4841] TS 1f5e4464cf2f4e57b977e56f15e323af (127.28.154.130:34865): tablet c1d8449979344cdcbc63b1a0961ae989 (table test-table [id=e2d76311f51a4ab094579348b31d3b99]) successfully deleted
I20260811 09:08:40.086663 11410 tablet_service.cc:1467] Processing CreateTablet for tablet f6ef2c2424dd4e49af7db09c1114908d (DEFAULT_TABLE table=test-table [id=26aa0a54b73347ff9b5658ca0c391075]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:08:40.086653 11697 tablet_service.cc:1467] Processing CreateTablet for tablet f6ef2c2424dd4e49af7db09c1114908d (DEFAULT_TABLE table=test-table [id=26aa0a54b73347ff9b5658ca0c391075]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:08:40.087143 11697 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f6ef2c2424dd4e49af7db09c1114908d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:08:40.087152 11410 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f6ef2c2424dd4e49af7db09c1114908d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:08:40.088367 12160 ts_tablet_manager.cc:1905] T c1d8449979344cdcbc63b1a0961ae989 P e6ba0c4707a9480b8309f094af83bd48: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:08:40.095628 12114 tablet_bootstrap.cc:492] T f6ef2c2424dd4e49af7db09c1114908d P 1f5e4464cf2f4e57b977e56f15e323af: Bootstrap starting.
I20260811 09:08:40.097738 12114 tablet_bootstrap.cc:654] T f6ef2c2424dd4e49af7db09c1114908d P 1f5e4464cf2f4e57b977e56f15e323af: Neither blocks nor log segments found. Creating new log.
I20260811 09:08:40.098657 12113 tablet_bootstrap.cc:492] T f6ef2c2424dd4e49af7db09c1114908d P b44912391db4452ba9f8e77aa97ff22b: Bootstrap starting.
I20260811 09:08:40.099627 12160 ts_tablet_manager.cc:1918] T c1d8449979344cdcbc63b1a0961ae989 P e6ba0c4707a9480b8309f094af83bd48: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:08:40.099924 12160 log.cc:1196] T c1d8449979344cdcbc63b1a0961ae989 P e6ba0c4707a9480b8309f094af83bd48: Deleting WAL directory at /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439250192320-29290-0/minicluster-data/ts-0/wal/wals/c1d8449979344cdcbc63b1a0961ae989
I20260811 09:08:40.100666 12119 tablet_bootstrap.cc:492] T f6ef2c2424dd4e49af7db09c1114908d P e6ba0c4707a9480b8309f094af83bd48: Bootstrap starting.
I20260811 09:08:40.102571 12119 tablet_bootstrap.cc:654] T f6ef2c2424dd4e49af7db09c1114908d P e6ba0c4707a9480b8309f094af83bd48: Neither blocks nor log segments found. Creating new log.
I20260811 09:08:40.102867 12113 tablet_bootstrap.cc:654] T f6ef2c2424dd4e49af7db09c1114908d P b44912391db4452ba9f8e77aa97ff22b: Neither blocks nor log segments found. Creating new log.
I20260811 09:08:40.107478 12160 ts_tablet_manager.cc:1939] T c1d8449979344cdcbc63b1a0961ae989 P e6ba0c4707a9480b8309f094af83bd48: Deleting consensus metadata
I20260811 09:08:40.108892 12113 tablet_bootstrap.cc:492] T f6ef2c2424dd4e49af7db09c1114908d P b44912391db4452ba9f8e77aa97ff22b: No bootstrap required, opened a new log
I20260811 09:08:40.109078 12113 ts_tablet_manager.cc:1397] T f6ef2c2424dd4e49af7db09c1114908d P b44912391db4452ba9f8e77aa97ff22b: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.004s
I20260811 09:08:40.109963 12113 raft_consensus.cc:348] T f6ef2c2424dd4e49af7db09c1114908d P b44912391db4452ba9f8e77aa97ff22b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6ba0c4707a9480b8309f094af83bd48" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41463 } } peers { permanent_uuid: "1f5e4464cf2f4e57b977e56f15e323af" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34865 } } peers { permanent_uuid: "b44912391db4452ba9f8e77aa97ff22b" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 39043 } }
I20260811 09:08:40.110164 11141 catalog_manager.cc:4841] TS e6ba0c4707a9480b8309f094af83bd48 (127.28.154.129:41463): tablet c1d8449979344cdcbc63b1a0961ae989 (table test-table [id=e2d76311f51a4ab094579348b31d3b99]) successfully deleted
I20260811 09:08:40.110227 12113 raft_consensus.cc:374] T f6ef2c2424dd4e49af7db09c1114908d P b44912391db4452ba9f8e77aa97ff22b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:08:40.110314 12113 raft_consensus.cc:729] T f6ef2c2424dd4e49af7db09c1114908d P b44912391db4452ba9f8e77aa97ff22b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b44912391db4452ba9f8e77aa97ff22b, State: Initialized, Role: FOLLOWER
I20260811 09:08:40.110699 12113 consensus_queue.cc:260] T f6ef2c2424dd4e49af7db09c1114908d P b44912391db4452ba9f8e77aa97ff22b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6ba0c4707a9480b8309f094af83bd48" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41463 } } peers { permanent_uuid: "1f5e4464cf2f4e57b977e56f15e323af" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34865 } } peers { permanent_uuid: "b44912391db4452ba9f8e77aa97ff22b" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 39043 } }
I20260811 09:08:40.113509 12119 tablet_bootstrap.cc:492] T f6ef2c2424dd4e49af7db09c1114908d P e6ba0c4707a9480b8309f094af83bd48: No bootstrap required, opened a new log
I20260811 09:08:40.113754 12119 ts_tablet_manager.cc:1397] T f6ef2c2424dd4e49af7db09c1114908d P e6ba0c4707a9480b8309f094af83bd48: Time spent bootstrapping tablet: real 0.013s user 0.006s sys 0.005s
I20260811 09:08:40.115134 12119 raft_consensus.cc:348] T f6ef2c2424dd4e49af7db09c1114908d P e6ba0c4707a9480b8309f094af83bd48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6ba0c4707a9480b8309f094af83bd48" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41463 } } peers { permanent_uuid: "1f5e4464cf2f4e57b977e56f15e323af" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34865 } } peers { permanent_uuid: "b44912391db4452ba9f8e77aa97ff22b" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 39043 } }
I20260811 09:08:40.115330 12113 ts_tablet_manager.cc:1428] T f6ef2c2424dd4e49af7db09c1114908d P b44912391db4452ba9f8e77aa97ff22b: Time spent starting tablet: real 0.006s user 0.000s sys 0.003s
I20260811 09:08:40.115511 12119 raft_consensus.cc:374] T f6ef2c2424dd4e49af7db09c1114908d P e6ba0c4707a9480b8309f094af83bd48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:08:40.115689 12119 raft_consensus.cc:729] T f6ef2c2424dd4e49af7db09c1114908d P e6ba0c4707a9480b8309f094af83bd48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6ba0c4707a9480b8309f094af83bd48, State: Initialized, Role: FOLLOWER
I20260811 09:08:40.116106 12119 consensus_queue.cc:260] T f6ef2c2424dd4e49af7db09c1114908d P e6ba0c4707a9480b8309f094af83bd48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6ba0c4707a9480b8309f094af83bd48" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41463 } } peers { permanent_uuid: "1f5e4464cf2f4e57b977e56f15e323af" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34865 } } peers { permanent_uuid: "b44912391db4452ba9f8e77aa97ff22b" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 39043 } }
I20260811 09:08:40.116894 12119 ts_tablet_manager.cc:1428] T f6ef2c2424dd4e49af7db09c1114908d P e6ba0c4707a9480b8309f094af83bd48: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260811 09:08:40.119410 12114 tablet_bootstrap.cc:492] T f6ef2c2424dd4e49af7db09c1114908d P 1f5e4464cf2f4e57b977e56f15e323af: No bootstrap required, opened a new log
I20260811 09:08:40.119594 12114 ts_tablet_manager.cc:1397] T f6ef2c2424dd4e49af7db09c1114908d P 1f5e4464cf2f4e57b977e56f15e323af: Time spent bootstrapping tablet: real 0.024s user 0.015s sys 0.004s
I20260811 09:08:40.120473 12114 raft_consensus.cc:348] T f6ef2c2424dd4e49af7db09c1114908d P 1f5e4464cf2f4e57b977e56f15e323af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6ba0c4707a9480b8309f094af83bd48" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41463 } } peers { permanent_uuid: "1f5e4464cf2f4e57b977e56f15e323af" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34865 } } peers { permanent_uuid: "b44912391db4452ba9f8e77aa97ff22b" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 39043 } }
I20260811 09:08:40.120725 12114 raft_consensus.cc:374] T f6ef2c2424dd4e49af7db09c1114908d P 1f5e4464cf2f4e57b977e56f15e323af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:08:40.120834 12114 raft_consensus.cc:729] T f6ef2c2424dd4e49af7db09c1114908d P 1f5e4464cf2f4e57b977e56f15e323af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f5e4464cf2f4e57b977e56f15e323af, State: Initialized, Role: FOLLOWER
I20260811 09:08:40.121171 12114 consensus_queue.cc:260] T f6ef2c2424dd4e49af7db09c1114908d P 1f5e4464cf2f4e57b977e56f15e323af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6ba0c4707a9480b8309f094af83bd48" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41463 } } peers { permanent_uuid: "1f5e4464cf2f4e57b977e56f15e323af" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34865 } } peers { permanent_uuid: "b44912391db4452ba9f8e77aa97ff22b" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 39043 } }
I20260811 09:08:40.121937 12114 ts_tablet_manager.cc:1428] T f6ef2c2424dd4e49af7db09c1114908d P 1f5e4464cf2f4e57b977e56f15e323af: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260811 09:08:40.277509 12121 raft_consensus.cc:482] T f6ef2c2424dd4e49af7db09c1114908d P b44912391db4452ba9f8e77aa97ff22b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:08:40.277796 12121 raft_consensus.cc:504] T f6ef2c2424dd4e49af7db09c1114908d P b44912391db4452ba9f8e77aa97ff22b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6ba0c4707a9480b8309f094af83bd48" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41463 } } peers { permanent_uuid: "1f5e4464cf2f4e57b977e56f15e323af" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34865 } } peers { permanent_uuid: "b44912391db4452ba9f8e77aa97ff22b" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 39043 } }
I20260811 09:08:40.279711 12121 leader_election.cc:288] T f6ef2c2424dd4e49af7db09c1114908d P b44912391db4452ba9f8e77aa97ff22b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e6ba0c4707a9480b8309f094af83bd48 (127.28.154.129:41463), 1f5e4464cf2f4e57b977e56f15e323af (127.28.154.130:34865)
I20260811 09:08:40.287765 12165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:40.280092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.129:41463 (local address 127.28.154.131:50751)
0811 09:08:40.280728 (+ 636us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:40.280748 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:40.280903 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:40.281503 (+ 600us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:40.281508 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:40.281544 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:08:40.282044 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:40.282072 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:40.283609 (+ 1537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:40.283614 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:40.285554 (+ 1940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:40.285586 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:40.286945 (+ 1359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:40.286952 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:40.287125 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:40.287152 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 09:08:40.287238 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 09:08:40.287369 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":514,"mutex_wait_us":1,"thread_start_us":350,"threads_started":1}
I20260811 09:08:40.287750 12112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:40.280083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.131:50751 (local address 127.28.154.129:41463)
0811 09:08:40.280319 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:40.280326 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:40.281013 (+ 687us) server_negotiation.cc:407] Connection header received
0811 09:08:40.281115 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:40.281121 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:40.281220 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:40.281319 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:08:40.282254 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:40.283408 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:40.285761 (+ 2353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:40.286757 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:40.286945 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:40.287029 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:40.287284 (+ 255us) server_negotiation.cc:299] Negotiation successful
0811 09:08:40.287415 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 09:08:40.288713 11448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6ef2c2424dd4e49af7db09c1114908d" candidate_uuid: "b44912391db4452ba9f8e77aa97ff22b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6ba0c4707a9480b8309f094af83bd48" is_pre_election: true
I20260811 09:08:40.289227 11448 raft_consensus.cc:2454] T f6ef2c2424dd4e49af7db09c1114908d P e6ba0c4707a9480b8309f094af83bd48 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b44912391db4452ba9f8e77aa97ff22b in term 0.
I20260811 09:08:40.289490 12166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:40.280925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.130:34865 (local address 127.28.154.131:57049)
0811 09:08:40.281991 (+ 1066us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:40.282010 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:40.282146 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:40.282585 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:40.282590 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:40.282618 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:08:40.283007 (+ 389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:40.283020 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:40.284489 (+ 1469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:40.284494 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:40.286546 (+ 2052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:40.286562 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:40.288732 (+ 2170us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:40.288738 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:40.288913 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:40.288947 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 09:08:40.289045 (+ 98us) client_negotiation.cc:240] Negotiation successful
0811 09:08:40.289182 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":954,"thread_start_us":729,"threads_started":1}
I20260811 09:08:40.289621 12103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:40.281256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.131:57049 (local address 127.28.154.130:34865)
0811 09:08:40.281458 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:40.281464 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:40.282103 (+ 639us) server_negotiation.cc:407] Connection header received
0811 09:08:40.282343 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:40.282350 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:40.282443 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:40.282533 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:08:40.283198 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:40.284278 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:40.286740 (+ 2462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:40.287976 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:40.288126 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:40.288197 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:40.289190 (+ 993us) server_negotiation.cc:299] Negotiation successful
0811 09:08:40.289339 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260811 09:08:40.290000 11846 leader_election.cc:302] T f6ef2c2424dd4e49af7db09c1114908d P b44912391db4452ba9f8e77aa97ff22b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b44912391db4452ba9f8e77aa97ff22b, e6ba0c4707a9480b8309f094af83bd48; no voters:
I20260811 09:08:40.290395 12121 raft_consensus.cc:2781] T f6ef2c2424dd4e49af7db09c1114908d P b44912391db4452ba9f8e77aa97ff22b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 09:08:40.290522 12121 raft_consensus.cc:482] T f6ef2c2424dd4e49af7db09c1114908d P b44912391db4452ba9f8e77aa97ff22b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 09:08:40.290606 12121 raft_consensus.cc:3037] T f6ef2c2424dd4e49af7db09c1114908d P b44912391db4452ba9f8e77aa97ff22b [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:08:40.290782 11737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6ef2c2424dd4e49af7db09c1114908d" candidate_uuid: "b44912391db4452ba9f8e77aa97ff22b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f5e4464cf2f4e57b977e56f15e323af" is_pre_election: true
I20260811 09:08:40.290984 11737 raft_consensus.cc:2454] T f6ef2c2424dd4e49af7db09c1114908d P 1f5e4464cf2f4e57b977e56f15e323af [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b44912391db4452ba9f8e77aa97ff22b in term 0.
I20260811 09:08:40.293915 12121 raft_consensus.cc:504] T f6ef2c2424dd4e49af7db09c1114908d P b44912391db4452ba9f8e77aa97ff22b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6ba0c4707a9480b8309f094af83bd48" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41463 } } peers { permanent_uuid: "1f5e4464cf2f4e57b977e56f15e323af" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34865 } } peers { permanent_uuid: "b44912391db4452ba9f8e77aa97ff22b" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 39043 } }
I20260811 09:08:40.294644 12121 leader_election.cc:288] T f6ef2c2424dd4e49af7db09c1114908d P b44912391db4452ba9f8e77aa97ff22b [CANDIDATE]: Term 1 election: Requested vote from peers e6ba0c4707a9480b8309f094af83bd48 (127.28.154.129:41463), 1f5e4464cf2f4e57b977e56f15e323af (127.28.154.130:34865)
I20260811 09:08:40.295089 11448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6ef2c2424dd4e49af7db09c1114908d" candidate_uuid: "b44912391db4452ba9f8e77aa97ff22b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6ba0c4707a9480b8309f094af83bd48"
I20260811 09:08:40.295209 11737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f6ef2c2424dd4e49af7db09c1114908d" candidate_uuid: "b44912391db4452ba9f8e77aa97ff22b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f5e4464cf2f4e57b977e56f15e323af"
I20260811 09:08:40.295336 11448 raft_consensus.cc:3037] T f6ef2c2424dd4e49af7db09c1114908d P e6ba0c4707a9480b8309f094af83bd48 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:08:40.295423 11737 raft_consensus.cc:3037] T f6ef2c2424dd4e49af7db09c1114908d P 1f5e4464cf2f4e57b977e56f15e323af [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:08:40.298094 11737 raft_consensus.cc:2454] T f6ef2c2424dd4e49af7db09c1114908d P 1f5e4464cf2f4e57b977e56f15e323af [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b44912391db4452ba9f8e77aa97ff22b in term 1.
I20260811 09:08:40.298524 11846 leader_election.cc:302] T f6ef2c2424dd4e49af7db09c1114908d P b44912391db4452ba9f8e77aa97ff22b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f5e4464cf2f4e57b977e56f15e323af, b44912391db4452ba9f8e77aa97ff22b; no voters:
I20260811 09:08:40.298810 12121 raft_consensus.cc:2781] T f6ef2c2424dd4e49af7db09c1114908d P b44912391db4452ba9f8e77aa97ff22b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:08:40.299011 11448 raft_consensus.cc:2454] T f6ef2c2424dd4e49af7db09c1114908d P e6ba0c4707a9480b8309f094af83bd48 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b44912391db4452ba9f8e77aa97ff22b in term 1.
I20260811 09:08:40.299371 12121 raft_consensus.cc:686] T f6ef2c2424dd4e49af7db09c1114908d P b44912391db4452ba9f8e77aa97ff22b [term 1 LEADER]: Becoming Leader. State: Replica: b44912391db4452ba9f8e77aa97ff22b, State: Running, Role: LEADER
I20260811 09:08:40.299822 12121 consensus_queue.cc:237] T f6ef2c2424dd4e49af7db09c1114908d P b44912391db4452ba9f8e77aa97ff22b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6ba0c4707a9480b8309f094af83bd48" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41463 } } peers { permanent_uuid: "1f5e4464cf2f4e57b977e56f15e323af" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34865 } } peers { permanent_uuid: "b44912391db4452ba9f8e77aa97ff22b" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 39043 } }
I20260811 09:08:40.303571 11167 catalog_manager.cc:5495] T f6ef2c2424dd4e49af7db09c1114908d P b44912391db4452ba9f8e77aa97ff22b reported cstate change: term changed from 0 to 1, leader changed from <none> to b44912391db4452ba9f8e77aa97ff22b (127.28.154.131). New cstate: current_term: 1 leader_uuid: "b44912391db4452ba9f8e77aa97ff22b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6ba0c4707a9480b8309f094af83bd48" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1f5e4464cf2f4e57b977e56f15e323af" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34865 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b44912391db4452ba9f8e77aa97ff22b" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 39043 } health_report { overall_health: HEALTHY } } }
Aug 11 09:08:40 dist-test-slave-xt4k krb5kdc[11078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439317, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.154.190@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM
I20260811 09:08:40.382210 12183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:40.362876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:45067 (local address 127.0.0.1:55720)
0811 09:08:40.363753 (+ 877us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:40.363773 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:40.363907 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:40.364266 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:40.364269 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:40.365185 (+ 916us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:08:40.365439 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:40.365450 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:40.367009 (+ 1559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:40.367013 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:40.367962 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:40.367975 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:40.368933 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:40.368938 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:40.369067 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:40.372860 (+ 3793us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:08:40.372881 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:08:40.375541 (+ 2660us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:08:40.379588 (+ 4047us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:08:40.379604 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:08:40.379611 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:08:40.380071 (+ 460us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:08:40.380550 (+ 479us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:08:40.380554 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:08:40.380556 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:08:40.380707 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:08:40.381330 (+ 623us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:08:40.381336 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:08:40.381492 (+ 156us) client_negotiation.cc:769] Sending connection context
0811 09:08:40.381642 (+ 150us) client_negotiation.cc:240] Negotiation successful
0811 09:08:40.381856 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":724,"thread_start_us":564,"threads_started":1}
I20260811 09:08:40.382611 12181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:40.362876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55720 (local address 127.28.154.190:45067)
0811 09:08:40.363543 (+ 667us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:40.363550 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:40.363870 (+ 320us) server_negotiation.cc:407] Connection header received
0811 09:08:40.364023 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:40.364029 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:40.364121 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:40.364256 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:40.365622 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:40.366826 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:40.368109 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:40.368766 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:40.368912 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:40.375800 (+ 6888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:40.375808 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:40.375813 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:08:40.375853 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:40.379423 (+ 3570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:40.380220 (+ 797us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:40.380225 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:40.380228 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:40.380430 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:40.380816 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:40.380822 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:40.380825 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:40.381195 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:40.381332 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:40.381771 (+ 439us) server_negotiation.cc:299] Negotiation successful
0811 09:08:40.382111 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":514,"thread_start_us":299,"threads_started":1}
I20260811 09:08:40.387992 11167 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.28.154.190@KRBTEST.COM'} at 127.0.0.1:55720:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 09:08:40.389139 11167 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 09:08:40.395888 11410 tablet_service.cc:1467] Processing CreateTablet for tablet 3b7282c772a84710956008e2651661f1 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=542dd4aa0e6b4827aa0d938c03a0e5ce]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 09:08:40.396562 11410 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b7282c772a84710956008e2651661f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:08:40.396446 11697 tablet_service.cc:1467] Processing CreateTablet for tablet 3b7282c772a84710956008e2651661f1 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=542dd4aa0e6b4827aa0d938c03a0e5ce]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 09:08:40.397020 11697 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b7282c772a84710956008e2651661f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:08:40.401325 11935 tablet_service.cc:1467] Processing CreateTablet for tablet 3b7282c772a84710956008e2651661f1 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=542dd4aa0e6b4827aa0d938c03a0e5ce]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 09:08:40.401912 11935 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3b7282c772a84710956008e2651661f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:08:40.404052 12119 tablet_bootstrap.cc:492] T 3b7282c772a84710956008e2651661f1 P e6ba0c4707a9480b8309f094af83bd48: Bootstrap starting.
I20260811 09:08:40.405131 12114 tablet_bootstrap.cc:492] T 3b7282c772a84710956008e2651661f1 P 1f5e4464cf2f4e57b977e56f15e323af: Bootstrap starting.
I20260811 09:08:40.405925 12119 tablet_bootstrap.cc:654] T 3b7282c772a84710956008e2651661f1 P e6ba0c4707a9480b8309f094af83bd48: Neither blocks nor log segments found. Creating new log.
I20260811 09:08:40.406373 12114 tablet_bootstrap.cc:654] T 3b7282c772a84710956008e2651661f1 P 1f5e4464cf2f4e57b977e56f15e323af: Neither blocks nor log segments found. Creating new log.
I20260811 09:08:40.408692 12119 tablet_bootstrap.cc:492] T 3b7282c772a84710956008e2651661f1 P e6ba0c4707a9480b8309f094af83bd48: No bootstrap required, opened a new log
I20260811 09:08:40.408896 12119 ts_tablet_manager.cc:1397] T 3b7282c772a84710956008e2651661f1 P e6ba0c4707a9480b8309f094af83bd48: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.001s
I20260811 09:08:40.409796 12119 raft_consensus.cc:348] T 3b7282c772a84710956008e2651661f1 P e6ba0c4707a9480b8309f094af83bd48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6ba0c4707a9480b8309f094af83bd48" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41463 } } peers { permanent_uuid: "1f5e4464cf2f4e57b977e56f15e323af" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34865 } } peers { permanent_uuid: "b44912391db4452ba9f8e77aa97ff22b" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 39043 } }
I20260811 09:08:40.410079 12119 raft_consensus.cc:374] T 3b7282c772a84710956008e2651661f1 P e6ba0c4707a9480b8309f094af83bd48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:08:40.410176 12119 raft_consensus.cc:729] T 3b7282c772a84710956008e2651661f1 P e6ba0c4707a9480b8309f094af83bd48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6ba0c4707a9480b8309f094af83bd48, State: Initialized, Role: FOLLOWER
I20260811 09:08:40.410490 12119 consensus_queue.cc:260] T 3b7282c772a84710956008e2651661f1 P e6ba0c4707a9480b8309f094af83bd48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6ba0c4707a9480b8309f094af83bd48" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41463 } } peers { permanent_uuid: "1f5e4464cf2f4e57b977e56f15e323af" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34865 } } peers { permanent_uuid: "b44912391db4452ba9f8e77aa97ff22b" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 39043 } }
I20260811 09:08:40.410969 12146 tablet_replica.cc:439] T 3b7282c772a84710956008e2651661f1 P e6ba0c4707a9480b8309f094af83bd48: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6ba0c4707a9480b8309f094af83bd48" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41463 } } peers { permanent_uuid: "1f5e4464cf2f4e57b977e56f15e323af" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34865 } } peers { permanent_uuid: "b44912391db4452ba9f8e77aa97ff22b" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 39043 } } }
I20260811 09:08:40.411492 12146 tablet_replica.cc:442] T 3b7282c772a84710956008e2651661f1 P e6ba0c4707a9480b8309f094af83bd48: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 09:08:40.411758 12113 tablet_bootstrap.cc:492] T 3b7282c772a84710956008e2651661f1 P b44912391db4452ba9f8e77aa97ff22b: Bootstrap starting.
I20260811 09:08:40.413681 12113 tablet_bootstrap.cc:654] T 3b7282c772a84710956008e2651661f1 P b44912391db4452ba9f8e77aa97ff22b: Neither blocks nor log segments found. Creating new log.
I20260811 09:08:40.414754 12119 ts_tablet_manager.cc:1428] T 3b7282c772a84710956008e2651661f1 P e6ba0c4707a9480b8309f094af83bd48: Time spent starting tablet: real 0.006s user 0.000s sys 0.002s
I20260811 09:08:40.420696 12114 tablet_bootstrap.cc:492] T 3b7282c772a84710956008e2651661f1 P 1f5e4464cf2f4e57b977e56f15e323af: No bootstrap required, opened a new log
I20260811 09:08:40.420894 12114 ts_tablet_manager.cc:1397] T 3b7282c772a84710956008e2651661f1 P 1f5e4464cf2f4e57b977e56f15e323af: Time spent bootstrapping tablet: real 0.016s user 0.001s sys 0.008s
I20260811 09:08:40.421692 12114 raft_consensus.cc:348] T 3b7282c772a84710956008e2651661f1 P 1f5e4464cf2f4e57b977e56f15e323af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6ba0c4707a9480b8309f094af83bd48" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41463 } } peers { permanent_uuid: "1f5e4464cf2f4e57b977e56f15e323af" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34865 } } peers { permanent_uuid: "b44912391db4452ba9f8e77aa97ff22b" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 39043 } }
I20260811 09:08:40.421945 12114 raft_consensus.cc:374] T 3b7282c772a84710956008e2651661f1 P 1f5e4464cf2f4e57b977e56f15e323af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:08:40.422024 12114 raft_consensus.cc:729] T 3b7282c772a84710956008e2651661f1 P 1f5e4464cf2f4e57b977e56f15e323af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f5e4464cf2f4e57b977e56f15e323af, State: Initialized, Role: FOLLOWER
I20260811 09:08:40.422278 12114 consensus_queue.cc:260] T 3b7282c772a84710956008e2651661f1 P 1f5e4464cf2f4e57b977e56f15e323af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6ba0c4707a9480b8309f094af83bd48" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41463 } } peers { permanent_uuid: "1f5e4464cf2f4e57b977e56f15e323af" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34865 } } peers { permanent_uuid: "b44912391db4452ba9f8e77aa97ff22b" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 39043 } }
I20260811 09:08:40.422792 12123 tablet_replica.cc:439] T 3b7282c772a84710956008e2651661f1 P 1f5e4464cf2f4e57b977e56f15e323af: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6ba0c4707a9480b8309f094af83bd48" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41463 } } peers { permanent_uuid: "1f5e4464cf2f4e57b977e56f15e323af" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34865 } } peers { permanent_uuid: "b44912391db4452ba9f8e77aa97ff22b" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 39043 } } }
I20260811 09:08:40.423141 12123 tablet_replica.cc:442] T 3b7282c772a84710956008e2651661f1 P 1f5e4464cf2f4e57b977e56f15e323af: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 09:08:40.423804 12114 ts_tablet_manager.cc:1428] T 3b7282c772a84710956008e2651661f1 P 1f5e4464cf2f4e57b977e56f15e323af: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260811 09:08:40.425279 12113 tablet_bootstrap.cc:492] T 3b7282c772a84710956008e2651661f1 P b44912391db4452ba9f8e77aa97ff22b: No bootstrap required, opened a new log
I20260811 09:08:40.425470 12113 ts_tablet_manager.cc:1397] T 3b7282c772a84710956008e2651661f1 P b44912391db4452ba9f8e77aa97ff22b: Time spent bootstrapping tablet: real 0.014s user 0.004s sys 0.000s
I20260811 09:08:40.426280 12113 raft_consensus.cc:348] T 3b7282c772a84710956008e2651661f1 P b44912391db4452ba9f8e77aa97ff22b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6ba0c4707a9480b8309f094af83bd48" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41463 } } peers { permanent_uuid: "1f5e4464cf2f4e57b977e56f15e323af" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34865 } } peers { permanent_uuid: "b44912391db4452ba9f8e77aa97ff22b" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 39043 } }
I20260811 09:08:40.426537 12113 raft_consensus.cc:374] T 3b7282c772a84710956008e2651661f1 P b44912391db4452ba9f8e77aa97ff22b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:08:40.426627 12113 raft_consensus.cc:729] T 3b7282c772a84710956008e2651661f1 P b44912391db4452ba9f8e77aa97ff22b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b44912391db4452ba9f8e77aa97ff22b, State: Initialized, Role: FOLLOWER
I20260811 09:08:40.426883 12113 consensus_queue.cc:260] T 3b7282c772a84710956008e2651661f1 P b44912391db4452ba9f8e77aa97ff22b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6ba0c4707a9480b8309f094af83bd48" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41463 } } peers { permanent_uuid: "1f5e4464cf2f4e57b977e56f15e323af" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34865 } } peers { permanent_uuid: "b44912391db4452ba9f8e77aa97ff22b" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 39043 } }
I20260811 09:08:40.427327 12121 tablet_replica.cc:439] T 3b7282c772a84710956008e2651661f1 P b44912391db4452ba9f8e77aa97ff22b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6ba0c4707a9480b8309f094af83bd48" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41463 } } peers { permanent_uuid: "1f5e4464cf2f4e57b977e56f15e323af" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34865 } } peers { permanent_uuid: "b44912391db4452ba9f8e77aa97ff22b" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 39043 } } }
I20260811 09:08:40.427510 12113 ts_tablet_manager.cc:1428] T 3b7282c772a84710956008e2651661f1 P b44912391db4452ba9f8e77aa97ff22b: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260811 09:08:40.427639 12121 tablet_replica.cc:442] T 3b7282c772a84710956008e2651661f1 P b44912391db4452ba9f8e77aa97ff22b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 09:08:40.639297 12123 raft_consensus.cc:482] T 3b7282c772a84710956008e2651661f1 P 1f5e4464cf2f4e57b977e56f15e323af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:08:40.639523 12123 raft_consensus.cc:504] T 3b7282c772a84710956008e2651661f1 P 1f5e4464cf2f4e57b977e56f15e323af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6ba0c4707a9480b8309f094af83bd48" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41463 } } peers { permanent_uuid: "1f5e4464cf2f4e57b977e56f15e323af" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34865 } } peers { permanent_uuid: "b44912391db4452ba9f8e77aa97ff22b" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 39043 } }
I20260811 09:08:40.641320 12123 leader_election.cc:288] T 3b7282c772a84710956008e2651661f1 P 1f5e4464cf2f4e57b977e56f15e323af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e6ba0c4707a9480b8309f094af83bd48 (127.28.154.129:41463), b44912391db4452ba9f8e77aa97ff22b (127.28.154.131:39043)
I20260811 09:08:40.649864 12194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:40.641841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.130:56777 (local address 127.28.154.131:39043)
0811 09:08:40.642668 (+ 827us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:40.642675 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:40.643182 (+ 507us) server_negotiation.cc:407] Connection header received
0811 09:08:40.643330 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:40.643334 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:40.643407 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:40.643488 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:08:40.644201 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:40.645245 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:40.647539 (+ 2294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:40.648816 (+ 1277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:40.648981 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:40.649057 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:40.649455 (+ 398us) server_negotiation.cc:299] Negotiation successful
0811 09:08:40.649589 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":712,"thread_start_us":607,"threads_started":1}
I20260811 09:08:40.649842 12112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:40.641740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.130:53909 (local address 127.28.154.129:41463)
0811 09:08:40.641950 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:40.641956 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:40.642433 (+ 477us) server_negotiation.cc:407] Connection header received
0811 09:08:40.642850 (+ 417us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:40.642868 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:40.642960 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:40.643041 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:08:40.643850 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:40.644913 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:40.647539 (+ 2626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:40.648816 (+ 1277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:40.648981 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:40.649057 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:40.649424 (+ 367us) server_negotiation.cc:299] Negotiation successful
0811 09:08:40.649572 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 09:08:40.650025 12193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:40.641596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.129:41463 (local address 127.28.154.130:53909)
0811 09:08:40.642285 (+ 689us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:40.642311 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:40.642566 (+ 255us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:40.643255 (+ 689us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:40.643260 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:40.643282 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:08:40.643655 (+ 373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:40.643668 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:40.645080 (+ 1412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:40.645084 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:40.647333 (+ 2249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:40.647351 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:40.649038 (+ 1687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:40.649045 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:40.649239 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:40.649277 (+ 38us) client_negotiation.cc:769] Sending connection context
0811 09:08:40.649380 (+ 103us) client_negotiation.cc:240] Negotiation successful
0811 09:08:40.649626 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":590,"mutex_wait_us":75,"thread_start_us":473,"threads_started":1}
I20260811 09:08:40.650384 12195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:40.641604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.131:39043 (local address 127.28.154.130:56777)
0811 09:08:40.643084 (+ 1480us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:40.643103 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:40.643223 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:40.643575 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:40.643580 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:40.643647 (+ 67us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:08:40.644022 (+ 375us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:40.644032 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:40.645390 (+ 1358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:40.645395 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:40.647395 (+ 2000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:40.647406 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:40.649076 (+ 1670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:40.649082 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:40.649274 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:40.649300 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 09:08:40.649380 (+ 80us) client_negotiation.cc:240] Negotiation successful
0811 09:08:40.649599 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1361,"mutex_wait_us":541,"thread_start_us":763,"threads_started":1}
I20260811 09:08:40.650619 11448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b7282c772a84710956008e2651661f1" candidate_uuid: "1f5e4464cf2f4e57b977e56f15e323af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6ba0c4707a9480b8309f094af83bd48" is_pre_election: true
I20260811 09:08:40.650885 11448 raft_consensus.cc:2454] T 3b7282c772a84710956008e2651661f1 P e6ba0c4707a9480b8309f094af83bd48 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f5e4464cf2f4e57b977e56f15e323af in term 0.
I20260811 09:08:40.651161 11973 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b7282c772a84710956008e2651661f1" candidate_uuid: "1f5e4464cf2f4e57b977e56f15e323af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b44912391db4452ba9f8e77aa97ff22b" is_pre_election: true
I20260811 09:08:40.651410 11973 raft_consensus.cc:2454] T 3b7282c772a84710956008e2651661f1 P b44912391db4452ba9f8e77aa97ff22b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f5e4464cf2f4e57b977e56f15e323af in term 0.
I20260811 09:08:40.651422 11594 leader_election.cc:302] T 3b7282c772a84710956008e2651661f1 P 1f5e4464cf2f4e57b977e56f15e323af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f5e4464cf2f4e57b977e56f15e323af, e6ba0c4707a9480b8309f094af83bd48; no voters:
I20260811 09:08:40.651854 12123 raft_consensus.cc:2781] T 3b7282c772a84710956008e2651661f1 P 1f5e4464cf2f4e57b977e56f15e323af [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 09:08:40.651994 12123 raft_consensus.cc:482] T 3b7282c772a84710956008e2651661f1 P 1f5e4464cf2f4e57b977e56f15e323af [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 09:08:40.652128 12123 raft_consensus.cc:3037] T 3b7282c772a84710956008e2651661f1 P 1f5e4464cf2f4e57b977e56f15e323af [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:08:40.654524 12123 raft_consensus.cc:504] T 3b7282c772a84710956008e2651661f1 P 1f5e4464cf2f4e57b977e56f15e323af [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6ba0c4707a9480b8309f094af83bd48" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41463 } } peers { permanent_uuid: "1f5e4464cf2f4e57b977e56f15e323af" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34865 } } peers { permanent_uuid: "b44912391db4452ba9f8e77aa97ff22b" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 39043 } }
I20260811 09:08:40.655081 12123 leader_election.cc:288] T 3b7282c772a84710956008e2651661f1 P 1f5e4464cf2f4e57b977e56f15e323af [CANDIDATE]: Term 1 election: Requested vote from peers e6ba0c4707a9480b8309f094af83bd48 (127.28.154.129:41463), b44912391db4452ba9f8e77aa97ff22b (127.28.154.131:39043)
I20260811 09:08:40.655584 11448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b7282c772a84710956008e2651661f1" candidate_uuid: "1f5e4464cf2f4e57b977e56f15e323af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6ba0c4707a9480b8309f094af83bd48"
I20260811 09:08:40.655596 11973 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3b7282c772a84710956008e2651661f1" candidate_uuid: "1f5e4464cf2f4e57b977e56f15e323af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b44912391db4452ba9f8e77aa97ff22b"
I20260811 09:08:40.655777 11973 raft_consensus.cc:3037] T 3b7282c772a84710956008e2651661f1 P b44912391db4452ba9f8e77aa97ff22b [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:08:40.655797 11448 raft_consensus.cc:3037] T 3b7282c772a84710956008e2651661f1 P e6ba0c4707a9480b8309f094af83bd48 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:08:40.657750 11448 raft_consensus.cc:2454] T 3b7282c772a84710956008e2651661f1 P e6ba0c4707a9480b8309f094af83bd48 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f5e4464cf2f4e57b977e56f15e323af in term 1.
I20260811 09:08:40.657816 11973 raft_consensus.cc:2454] T 3b7282c772a84710956008e2651661f1 P b44912391db4452ba9f8e77aa97ff22b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f5e4464cf2f4e57b977e56f15e323af in term 1.
I20260811 09:08:40.658159 11594 leader_election.cc:302] T 3b7282c772a84710956008e2651661f1 P 1f5e4464cf2f4e57b977e56f15e323af [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f5e4464cf2f4e57b977e56f15e323af, e6ba0c4707a9480b8309f094af83bd48; no voters:
I20260811 09:08:40.658442 12123 raft_consensus.cc:2781] T 3b7282c772a84710956008e2651661f1 P 1f5e4464cf2f4e57b977e56f15e323af [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:08:40.658910 12123 raft_consensus.cc:686] T 3b7282c772a84710956008e2651661f1 P 1f5e4464cf2f4e57b977e56f15e323af [term 1 LEADER]: Becoming Leader. State: Replica: 1f5e4464cf2f4e57b977e56f15e323af, State: Running, Role: LEADER
I20260811 09:08:40.659279 12123 consensus_queue.cc:237] T 3b7282c772a84710956008e2651661f1 P 1f5e4464cf2f4e57b977e56f15e323af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6ba0c4707a9480b8309f094af83bd48" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41463 } } peers { permanent_uuid: "1f5e4464cf2f4e57b977e56f15e323af" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34865 } } peers { permanent_uuid: "b44912391db4452ba9f8e77aa97ff22b" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 39043 } }
I20260811 09:08:40.661159 12197 tablet_replica.cc:439] T 3b7282c772a84710956008e2651661f1 P 1f5e4464cf2f4e57b977e56f15e323af: TxnStatusTablet state changed. Reason: New leader 1f5e4464cf2f4e57b977e56f15e323af. Latest consensus state: current_term: 1 leader_uuid: "1f5e4464cf2f4e57b977e56f15e323af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6ba0c4707a9480b8309f094af83bd48" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41463 } } peers { permanent_uuid: "1f5e4464cf2f4e57b977e56f15e323af" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34865 } } peers { permanent_uuid: "b44912391db4452ba9f8e77aa97ff22b" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 39043 } } }
I20260811 09:08:40.661599 12197 tablet_replica.cc:442] T 3b7282c772a84710956008e2651661f1 P 1f5e4464cf2f4e57b977e56f15e323af: This TxnStatusTablet replica's current role is: LEADER
I20260811 09:08:40.662597 12200 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 09:08:40.662936 12200 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 09:08:40.663044 11167 catalog_manager.cc:5495] T 3b7282c772a84710956008e2651661f1 P 1f5e4464cf2f4e57b977e56f15e323af reported cstate change: term changed from 0 to 1, leader changed from <none> to 1f5e4464cf2f4e57b977e56f15e323af (127.28.154.130). New cstate: current_term: 1 leader_uuid: "1f5e4464cf2f4e57b977e56f15e323af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6ba0c4707a9480b8309f094af83bd48" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1f5e4464cf2f4e57b977e56f15e323af" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34865 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b44912391db4452ba9f8e77aa97ff22b" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 39043 } health_report { overall_health: UNKNOWN } } }
I20260811 09:08:40.694078 12121 consensus_queue.cc:1035] T f6ef2c2424dd4e49af7db09c1114908d P b44912391db4452ba9f8e77aa97ff22b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f5e4464cf2f4e57b977e56f15e323af" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34865 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:08:40.701650 12121 consensus_queue.cc:1035] T f6ef2c2424dd4e49af7db09c1114908d P b44912391db4452ba9f8e77aa97ff22b [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6ba0c4707a9480b8309f094af83bd48" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:08:40.723731 12103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:40.717159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42756 (local address 127.28.154.130:34865)
0811 09:08:40.717357 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:40.717364 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:40.717538 (+ 174us) server_negotiation.cc:407] Connection header received
0811 09:08:40.717840 (+ 302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:40.717845 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:40.717956 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:40.718091 (+ 135us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:08:40.718548 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:40.719357 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:40.721283 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:40.721784 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:40.721904 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:40.722497 (+ 593us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:08:40.723090 (+ 593us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:08:40.723202 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:40.723401 (+ 199us) server_negotiation.cc:299] Negotiation successful
0811 09:08:40.723480 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 09:08:40.723902 12183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:40.717139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.130:34865 (local address 127.0.0.1:42756)
0811 09:08:40.717416 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:40.717437 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:40.717593 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:40.718148 (+ 555us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:40.718153 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:40.718168 (+ 15us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 09:08:40.718376 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:40.718388 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:40.719491 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:40.719496 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:40.721134 (+ 1638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:40.721150 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:40.721953 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:40.721959 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:40.722213 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:40.722294 (+ 81us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 09:08:40.723252 (+ 958us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 09:08:40.723278 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 09:08:40.723375 (+ 97us) client_negotiation.cc:240] Negotiation successful
0811 09:08:40.723561 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260811 09:08:40.728600 11448 raft_consensus.cc:1261] T 3b7282c772a84710956008e2651661f1 P e6ba0c4707a9480b8309f094af83bd48 [term 1 FOLLOWER]: Refusing update from remote peer 1f5e4464cf2f4e57b977e56f15e323af: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:08:40.728593 11973 raft_consensus.cc:1261] T 3b7282c772a84710956008e2651661f1 P b44912391db4452ba9f8e77aa97ff22b [term 1 FOLLOWER]: Refusing update from remote peer 1f5e4464cf2f4e57b977e56f15e323af: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:08:40.729246 12123 consensus_queue.cc:1035] T 3b7282c772a84710956008e2651661f1 P 1f5e4464cf2f4e57b977e56f15e323af [LEADER]: Connected to new peer: Peer: permanent_uuid: "b44912391db4452ba9f8e77aa97ff22b" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 39043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:08:40.729540 12197 consensus_queue.cc:1035] T 3b7282c772a84710956008e2651661f1 P 1f5e4464cf2f4e57b977e56f15e323af [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6ba0c4707a9480b8309f094af83bd48" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:08:40.733323 12146 tablet_replica.cc:439] T 3b7282c772a84710956008e2651661f1 P e6ba0c4707a9480b8309f094af83bd48: TxnStatusTablet state changed. Reason: New leader 1f5e4464cf2f4e57b977e56f15e323af. Latest consensus state: current_term: 1 leader_uuid: "1f5e4464cf2f4e57b977e56f15e323af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6ba0c4707a9480b8309f094af83bd48" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41463 } } peers { permanent_uuid: "1f5e4464cf2f4e57b977e56f15e323af" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34865 } } peers { permanent_uuid: "b44912391db4452ba9f8e77aa97ff22b" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 39043 } } }
I20260811 09:08:40.733644 12146 tablet_replica.cc:442] T 3b7282c772a84710956008e2651661f1 P e6ba0c4707a9480b8309f094af83bd48: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 09:08:40.736395 12211 mvcc.cc:205] Tried to move back new op lower bound from 7317255457695682560 to 7317255457423826944. Current Snapshot: MvccSnapshot[applied={T|T < 7317255457695682560}]
I20260811 09:08:40.736917 12197 tablet_replica.cc:439] T 3b7282c772a84710956008e2651661f1 P 1f5e4464cf2f4e57b977e56f15e323af: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1f5e4464cf2f4e57b977e56f15e323af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6ba0c4707a9480b8309f094af83bd48" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41463 } } peers { permanent_uuid: "1f5e4464cf2f4e57b977e56f15e323af" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34865 } } peers { permanent_uuid: "b44912391db4452ba9f8e77aa97ff22b" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 39043 } } }
I20260811 09:08:40.737201 12197 tablet_replica.cc:442] T 3b7282c772a84710956008e2651661f1 P 1f5e4464cf2f4e57b977e56f15e323af: This TxnStatusTablet replica's current role is: LEADER
I20260811 09:08:40.737339 12216 mvcc.cc:205] Tried to move back new op lower bound from 7317255457695682560 to 7317255457423826944. Current Snapshot: MvccSnapshot[applied={T|T < 7317255457695682560}]
I20260811 09:08:40.739485 12146 tablet_replica.cc:439] T 3b7282c772a84710956008e2651661f1 P e6ba0c4707a9480b8309f094af83bd48: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1f5e4464cf2f4e57b977e56f15e323af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6ba0c4707a9480b8309f094af83bd48" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41463 } } peers { permanent_uuid: "1f5e4464cf2f4e57b977e56f15e323af" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34865 } } peers { permanent_uuid: "b44912391db4452ba9f8e77aa97ff22b" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 39043 } } }
I20260811 09:08:40.739693 12146 tablet_replica.cc:442] T 3b7282c772a84710956008e2651661f1 P e6ba0c4707a9480b8309f094af83bd48: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 09:08:40.743359 12121 tablet_replica.cc:439] T 3b7282c772a84710956008e2651661f1 P b44912391db4452ba9f8e77aa97ff22b: TxnStatusTablet state changed. Reason: New leader 1f5e4464cf2f4e57b977e56f15e323af. Latest consensus state: current_term: 1 leader_uuid: "1f5e4464cf2f4e57b977e56f15e323af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6ba0c4707a9480b8309f094af83bd48" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41463 } } peers { permanent_uuid: "1f5e4464cf2f4e57b977e56f15e323af" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34865 } } peers { permanent_uuid: "b44912391db4452ba9f8e77aa97ff22b" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 39043 } } }
I20260811 09:08:40.743696 12121 tablet_replica.cc:442] T 3b7282c772a84710956008e2651661f1 P b44912391db4452ba9f8e77aa97ff22b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 09:08:40.744123 12123 tablet_replica.cc:439] T 3b7282c772a84710956008e2651661f1 P 1f5e4464cf2f4e57b977e56f15e323af: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1f5e4464cf2f4e57b977e56f15e323af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6ba0c4707a9480b8309f094af83bd48" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41463 } } peers { permanent_uuid: "1f5e4464cf2f4e57b977e56f15e323af" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34865 } } peers { permanent_uuid: "b44912391db4452ba9f8e77aa97ff22b" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 39043 } } }
I20260811 09:08:40.744416 12123 tablet_replica.cc:442] T 3b7282c772a84710956008e2651661f1 P 1f5e4464cf2f4e57b977e56f15e323af: This TxnStatusTablet replica's current role is: LEADER
I20260811 09:08:40.748941 12121 tablet_replica.cc:439] T 3b7282c772a84710956008e2651661f1 P b44912391db4452ba9f8e77aa97ff22b: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1f5e4464cf2f4e57b977e56f15e323af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6ba0c4707a9480b8309f094af83bd48" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41463 } } peers { permanent_uuid: "1f5e4464cf2f4e57b977e56f15e323af" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34865 } } peers { permanent_uuid: "b44912391db4452ba9f8e77aa97ff22b" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 39043 } } }
I20260811 09:08:40.749270 12121 tablet_replica.cc:442] T 3b7282c772a84710956008e2651661f1 P b44912391db4452ba9f8e77aa97ff22b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 09:08:40.756285 12194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:40.746626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40642 (local address 127.28.154.131:39043)
0811 09:08:40.746804 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:40.746811 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:40.749998 (+ 3187us) server_negotiation.cc:407] Connection header received
0811 09:08:40.750329 (+ 331us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:40.750335 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:40.750459 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:40.750599 (+ 140us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:08:40.751240 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:40.752051 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:40.754049 (+ 1998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:40.754483 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:40.754579 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:40.755054 (+ 475us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:08:40.755658 (+ 604us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:08:40.755784 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:40.755946 (+ 162us) server_negotiation.cc:299] Negotiation successful
0811 09:08:40.756021 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 09:08:40.774947 12103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:40.768289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.131:43311 (local address 127.28.154.130:34865)
0811 09:08:40.768535 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:40.768542 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:40.769050 (+ 508us) server_negotiation.cc:407] Connection header received
0811 09:08:40.769232 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:40.769238 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:40.769335 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:40.769456 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:08:40.769976 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:40.771137 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:40.772660 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:40.773481 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:40.773611 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:40.774132 (+ 521us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:08:40.774247 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:08:40.774353 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:40.774492 (+ 139us) server_negotiation.cc:299] Negotiation successful
0811 09:08:40.774599 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260811 09:08:40.774981 12235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:40.768240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.130:34865 (local address 127.28.154.131:43311)
0811 09:08:40.768944 (+ 704us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:40.768962 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:40.769107 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:40.769455 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:40.769459 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:40.769468 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 09:08:40.769812 (+ 344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:40.769822 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:40.771299 (+ 1477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:40.771304 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:40.772525 (+ 1221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:40.772539 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:40.773712 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:40.773717 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:40.773867 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:40.773935 (+ 68us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 09:08:40.774367 (+ 432us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 09:08:40.774381 (+ 14us) client_negotiation.cc:769] Sending connection context
0811 09:08:40.774454 (+ 73us) client_negotiation.cc:240] Negotiation successful
0811 09:08:40.774612 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":543,"thread_start_us":390,"threads_started":1}
I20260811 09:08:40.846195 12194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:40.837711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.130:43705 (local address 127.28.154.131:39043)
0811 09:08:40.837901 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:40.837908 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:40.838371 (+ 463us) server_negotiation.cc:407] Connection header received
0811 09:08:40.838549 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:40.838556 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:40.838673 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:40.838767 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:08:40.839794 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:40.841041 (+ 1247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:40.843518 (+ 2477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:40.844294 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:40.844455 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:40.845318 (+ 863us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:08:40.845482 (+ 164us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:08:40.845549 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:40.845805 (+ 256us) server_negotiation.cc:299] Negotiation successful
0811 09:08:40.845897 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 09:08:40.846674 12244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:40.837580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.131:39043 (local address 127.28.154.130:43705)
0811 09:08:40.838263 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:40.838287 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:40.838416 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:40.839125 (+ 709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:40.839131 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:40.839149 (+ 18us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 09:08:40.839552 (+ 403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:40.839569 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:40.841312 (+ 1743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:40.841318 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:40.843332 (+ 2014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:40.843350 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:40.844687 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:40.844694 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:40.845008 (+ 314us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:40.845098 (+ 90us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 09:08:40.845637 (+ 539us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 09:08:40.845647 (+ 10us) client_negotiation.cc:769] Sending connection context
0811 09:08:40.845758 (+ 111us) client_negotiation.cc:240] Negotiation successful
0811 09:08:40.845916 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":325,"threads_started":1}
I20260811 09:08:40.899574 11167 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55718:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:08:40.899844 11167 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55718:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:08:40.903682 11167 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 32c27303c7e1421abd79752385c82cd5: Sending DeleteTablet for 3 replicas of tablet f6ef2c2424dd4e49af7db09c1114908d
I20260811 09:08:40.904776 11935 tablet_service.cc:1514] Processing DeleteTablet for tablet f6ef2c2424dd4e49af7db09c1114908d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:08:40 UTC) from {username='oryx', principal='oryx/127.28.154.190@KRBTEST.COM'} at 127.0.0.1:40632
I20260811 09:08:40.905280 11410 tablet_service.cc:1514] Processing DeleteTablet for tablet f6ef2c2424dd4e49af7db09c1114908d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:08:40 UTC) from {username='oryx', principal='oryx/127.28.154.190@KRBTEST.COM'} at 127.0.0.1:41522
I20260811 09:08:40.905754 11697 tablet_service.cc:1514] Processing DeleteTablet for tablet f6ef2c2424dd4e49af7db09c1114908d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:08:40 UTC) from {username='oryx', principal='oryx/127.28.154.190@KRBTEST.COM'} at 127.0.0.1:42752
I20260811 09:08:40.916186 12249 tablet_replica.cc:330] T f6ef2c2424dd4e49af7db09c1114908d P b44912391db4452ba9f8e77aa97ff22b: stopping tablet replica
I20260811 09:08:40.916590 12249 raft_consensus.cc:2229] T f6ef2c2424dd4e49af7db09c1114908d P b44912391db4452ba9f8e77aa97ff22b [term 1 LEADER]: Raft consensus shutting down.
I20260811 09:08:40.917361 12249 raft_consensus.cc:2258] T f6ef2c2424dd4e49af7db09c1114908d P b44912391db4452ba9f8e77aa97ff22b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:08:40.921226 12249 ts_tablet_manager.cc:1905] T f6ef2c2424dd4e49af7db09c1114908d P b44912391db4452ba9f8e77aa97ff22b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:08:40.928529 12249 ts_tablet_manager.cc:1918] T f6ef2c2424dd4e49af7db09c1114908d P b44912391db4452ba9f8e77aa97ff22b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 09:08:40.928690 12249 log.cc:1196] T f6ef2c2424dd4e49af7db09c1114908d P b44912391db4452ba9f8e77aa97ff22b: Deleting WAL directory at /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439250192320-29290-0/minicluster-data/ts-2/wal/wals/f6ef2c2424dd4e49af7db09c1114908d
I20260811 09:08:40.929147 12249 ts_tablet_manager.cc:1939] T f6ef2c2424dd4e49af7db09c1114908d P b44912391db4452ba9f8e77aa97ff22b: Deleting consensus metadata
I20260811 09:08:40.930393 11142 catalog_manager.cc:4841] TS b44912391db4452ba9f8e77aa97ff22b (127.28.154.131:39043): tablet f6ef2c2424dd4e49af7db09c1114908d (table test-table [id=26aa0a54b73347ff9b5658ca0c391075]) successfully deleted
I20260811 09:08:40.931639 12254 tablet_replica.cc:330] T f6ef2c2424dd4e49af7db09c1114908d P e6ba0c4707a9480b8309f094af83bd48: stopping tablet replica
I20260811 09:08:40.932022 12255 tablet_replica.cc:330] T f6ef2c2424dd4e49af7db09c1114908d P 1f5e4464cf2f4e57b977e56f15e323af: stopping tablet replica
I20260811 09:08:40.932233 12255 raft_consensus.cc:2229] T f6ef2c2424dd4e49af7db09c1114908d P 1f5e4464cf2f4e57b977e56f15e323af [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:08:40.932545 12255 raft_consensus.cc:2258] T f6ef2c2424dd4e49af7db09c1114908d P 1f5e4464cf2f4e57b977e56f15e323af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:08:40.932938 12254 raft_consensus.cc:2229] T f6ef2c2424dd4e49af7db09c1114908d P e6ba0c4707a9480b8309f094af83bd48 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:08:40.933171 12254 raft_consensus.cc:2258] T f6ef2c2424dd4e49af7db09c1114908d P e6ba0c4707a9480b8309f094af83bd48 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:08:40.942552 12254 ts_tablet_manager.cc:1905] T f6ef2c2424dd4e49af7db09c1114908d P e6ba0c4707a9480b8309f094af83bd48: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:08:40.942126 12255 ts_tablet_manager.cc:1905] T f6ef2c2424dd4e49af7db09c1114908d P 1f5e4464cf2f4e57b977e56f15e323af: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:08:40.950680 12254 ts_tablet_manager.cc:1918] T f6ef2c2424dd4e49af7db09c1114908d P e6ba0c4707a9480b8309f094af83bd48: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 09:08:40.950834 12254 log.cc:1196] T f6ef2c2424dd4e49af7db09c1114908d P e6ba0c4707a9480b8309f094af83bd48: Deleting WAL directory at /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439250192320-29290-0/minicluster-data/ts-0/wal/wals/f6ef2c2424dd4e49af7db09c1114908d
I20260811 09:08:40.951273 12254 ts_tablet_manager.cc:1939] T f6ef2c2424dd4e49af7db09c1114908d P e6ba0c4707a9480b8309f094af83bd48: Deleting consensus metadata
I20260811 09:08:40.952512 11141 catalog_manager.cc:4841] TS e6ba0c4707a9480b8309f094af83bd48 (127.28.154.129:41463): tablet f6ef2c2424dd4e49af7db09c1114908d (table test-table [id=26aa0a54b73347ff9b5658ca0c391075]) successfully deleted
I20260811 09:08:40.959225 12255 ts_tablet_manager.cc:1918] T f6ef2c2424dd4e49af7db09c1114908d P 1f5e4464cf2f4e57b977e56f15e323af: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 09:08:40.959391 12255 log.cc:1196] T f6ef2c2424dd4e49af7db09c1114908d P 1f5e4464cf2f4e57b977e56f15e323af: Deleting WAL directory at /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439250192320-29290-0/minicluster-data/ts-1/wal/wals/f6ef2c2424dd4e49af7db09c1114908d
I20260811 09:08:40.959910 12255 ts_tablet_manager.cc:1939] T f6ef2c2424dd4e49af7db09c1114908d P 1f5e4464cf2f4e57b977e56f15e323af: Deleting consensus metadata
I20260811 09:08:40.967109 11141 catalog_manager.cc:4841] TS 1f5e4464cf2f4e57b977e56f15e323af (127.28.154.130:34865): tablet f6ef2c2424dd4e49af7db09c1114908d (table test-table [id=26aa0a54b73347ff9b5658ca0c391075]) successfully deleted
Aug 11 09:08:40 dist-test-slave-xt4k krb5kdc[11078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439320, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 09:08:41 dist-test-slave-xt4k krb5kdc[11078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.28.154.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:08:41 dist-test-slave-xt4k krb5kdc[11078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.28.154.190@KRBTEST.COM, Server not found in Kerberos database
W20260811 09:08:41.047504 12264 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:08:41.022315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55726 (local address 127.28.154.190:45067)
0811 09:08:41.033896 (+ 11581us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:41.033903 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:41.033927 (+ 24us) server_negotiation.cc:407] Connection header received
0811 09:08:41.034000 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:41.034006 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:41.034095 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:41.034211 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:41.036150 (+ 1939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:41.037574 (+ 1424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:41.039427 (+ 1853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:41.040088 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:41.040231 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:41.047206 (+ 6975us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55726: BlockingRecv error: recv got EOF from 127.0.0.1:55726 (error 108)
Metrics: {"server-negotiator.queue_time_us":11424,"thread_start_us":11538,"threads_started":1}
Aug 11 09:08:41 dist-test-slave-xt4k krb5kdc[11078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439320, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM
I20260811 09:08:41.071426 12264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:41.051337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55732 (local address 127.28.154.190:45067)
0811 09:08:41.051536 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:41.051542 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:41.053007 (+ 1465us) server_negotiation.cc:407] Connection header received
0811 09:08:41.053211 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:41.053217 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:41.053312 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:41.053426 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:41.055001 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:41.056160 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:41.058122 (+ 1962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:41.058790 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:41.058931 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:41.064236 (+ 5305us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:41.064245 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:41.064250 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:08:41.064304 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:41.068037 (+ 3733us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:41.069020 (+ 983us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:41.069027 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:41.069031 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:41.069199 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:41.069666 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:41.069671 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:41.069675 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:41.070080 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:41.070351 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:41.070722 (+ 371us) server_negotiation.cc:299] Negotiation successful
0811 09:08:41.071059 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260811 09:08:41.076403 11167 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55732:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 09:08:41.077185 11167 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 09:08:41.085872 11410 tablet_service.cc:1467] Processing CreateTablet for tablet 23ed736026e04b7a8842146294da3a96 (DEFAULT_TABLE table=test-table [id=24a0754c020045bbb1cd06bd70c08d7b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:08:41.085995 11935 tablet_service.cc:1467] Processing CreateTablet for tablet 23ed736026e04b7a8842146294da3a96 (DEFAULT_TABLE table=test-table [id=24a0754c020045bbb1cd06bd70c08d7b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:08:41.086505 11410 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 23ed736026e04b7a8842146294da3a96. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:08:41.086690 11935 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 23ed736026e04b7a8842146294da3a96. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:08:41.089149 11697 tablet_service.cc:1467] Processing CreateTablet for tablet 23ed736026e04b7a8842146294da3a96 (DEFAULT_TABLE table=test-table [id=24a0754c020045bbb1cd06bd70c08d7b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:08:41.089593 11697 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 23ed736026e04b7a8842146294da3a96. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:08:41.101642 12274 tablet_bootstrap.cc:492] T 23ed736026e04b7a8842146294da3a96 P b44912391db4452ba9f8e77aa97ff22b: Bootstrap starting.
I20260811 09:08:41.105252 12274 tablet_bootstrap.cc:654] T 23ed736026e04b7a8842146294da3a96 P b44912391db4452ba9f8e77aa97ff22b: Neither blocks nor log segments found. Creating new log.
I20260811 09:08:41.112942 12280 tablet_bootstrap.cc:492] T 23ed736026e04b7a8842146294da3a96 P e6ba0c4707a9480b8309f094af83bd48: Bootstrap starting.
I20260811 09:08:41.114816 12280 tablet_bootstrap.cc:654] T 23ed736026e04b7a8842146294da3a96 P e6ba0c4707a9480b8309f094af83bd48: Neither blocks nor log segments found. Creating new log.
I20260811 09:08:41.114984 12276 tablet_bootstrap.cc:492] T 23ed736026e04b7a8842146294da3a96 P 1f5e4464cf2f4e57b977e56f15e323af: Bootstrap starting.
I20260811 09:08:41.114863 12274 tablet_bootstrap.cc:492] T 23ed736026e04b7a8842146294da3a96 P b44912391db4452ba9f8e77aa97ff22b: No bootstrap required, opened a new log
I20260811 09:08:41.115396 12274 ts_tablet_manager.cc:1397] T 23ed736026e04b7a8842146294da3a96 P b44912391db4452ba9f8e77aa97ff22b: Time spent bootstrapping tablet: real 0.014s user 0.007s sys 0.004s
I20260811 09:08:41.116384 12274 raft_consensus.cc:348] T 23ed736026e04b7a8842146294da3a96 P b44912391db4452ba9f8e77aa97ff22b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6ba0c4707a9480b8309f094af83bd48" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41463 } } peers { permanent_uuid: "1f5e4464cf2f4e57b977e56f15e323af" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34865 } } peers { permanent_uuid: "b44912391db4452ba9f8e77aa97ff22b" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 39043 } }
I20260811 09:08:41.116850 12274 raft_consensus.cc:374] T 23ed736026e04b7a8842146294da3a96 P b44912391db4452ba9f8e77aa97ff22b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:08:41.116995 12274 raft_consensus.cc:729] T 23ed736026e04b7a8842146294da3a96 P b44912391db4452ba9f8e77aa97ff22b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b44912391db4452ba9f8e77aa97ff22b, State: Initialized, Role: FOLLOWER
I20260811 09:08:41.117347 12274 consensus_queue.cc:260] T 23ed736026e04b7a8842146294da3a96 P b44912391db4452ba9f8e77aa97ff22b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6ba0c4707a9480b8309f094af83bd48" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41463 } } peers { permanent_uuid: "1f5e4464cf2f4e57b977e56f15e323af" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34865 } } peers { permanent_uuid: "b44912391db4452ba9f8e77aa97ff22b" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 39043 } }
I20260811 09:08:41.119194 12274 ts_tablet_manager.cc:1428] T 23ed736026e04b7a8842146294da3a96 P b44912391db4452ba9f8e77aa97ff22b: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260811 09:08:41.120849 12276 tablet_bootstrap.cc:654] T 23ed736026e04b7a8842146294da3a96 P 1f5e4464cf2f4e57b977e56f15e323af: Neither blocks nor log segments found. Creating new log.
I20260811 09:08:41.123766 12280 tablet_bootstrap.cc:492] T 23ed736026e04b7a8842146294da3a96 P e6ba0c4707a9480b8309f094af83bd48: No bootstrap required, opened a new log
I20260811 09:08:41.123940 12280 ts_tablet_manager.cc:1397] T 23ed736026e04b7a8842146294da3a96 P e6ba0c4707a9480b8309f094af83bd48: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.000s
I20260811 09:08:41.124802 12280 raft_consensus.cc:348] T 23ed736026e04b7a8842146294da3a96 P e6ba0c4707a9480b8309f094af83bd48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6ba0c4707a9480b8309f094af83bd48" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41463 } } peers { permanent_uuid: "1f5e4464cf2f4e57b977e56f15e323af" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34865 } } peers { permanent_uuid: "b44912391db4452ba9f8e77aa97ff22b" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 39043 } }
I20260811 09:08:41.125070 12280 raft_consensus.cc:374] T 23ed736026e04b7a8842146294da3a96 P e6ba0c4707a9480b8309f094af83bd48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:08:41.125173 12280 raft_consensus.cc:729] T 23ed736026e04b7a8842146294da3a96 P e6ba0c4707a9480b8309f094af83bd48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6ba0c4707a9480b8309f094af83bd48, State: Initialized, Role: FOLLOWER
I20260811 09:08:41.125517 12280 consensus_queue.cc:260] T 23ed736026e04b7a8842146294da3a96 P e6ba0c4707a9480b8309f094af83bd48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6ba0c4707a9480b8309f094af83bd48" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41463 } } peers { permanent_uuid: "1f5e4464cf2f4e57b977e56f15e323af" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34865 } } peers { permanent_uuid: "b44912391db4452ba9f8e77aa97ff22b" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 39043 } }
I20260811 09:08:41.125988 12276 tablet_bootstrap.cc:492] T 23ed736026e04b7a8842146294da3a96 P 1f5e4464cf2f4e57b977e56f15e323af: No bootstrap required, opened a new log
I20260811 09:08:41.126204 12276 ts_tablet_manager.cc:1397] T 23ed736026e04b7a8842146294da3a96 P 1f5e4464cf2f4e57b977e56f15e323af: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.000s
I20260811 09:08:41.127167 12276 raft_consensus.cc:348] T 23ed736026e04b7a8842146294da3a96 P 1f5e4464cf2f4e57b977e56f15e323af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6ba0c4707a9480b8309f094af83bd48" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41463 } } peers { permanent_uuid: "1f5e4464cf2f4e57b977e56f15e323af" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34865 } } peers { permanent_uuid: "b44912391db4452ba9f8e77aa97ff22b" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 39043 } }
I20260811 09:08:41.127518 12276 raft_consensus.cc:374] T 23ed736026e04b7a8842146294da3a96 P 1f5e4464cf2f4e57b977e56f15e323af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:08:41.127676 12276 raft_consensus.cc:729] T 23ed736026e04b7a8842146294da3a96 P 1f5e4464cf2f4e57b977e56f15e323af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f5e4464cf2f4e57b977e56f15e323af, State: Initialized, Role: FOLLOWER
I20260811 09:08:41.128057 12276 consensus_queue.cc:260] T 23ed736026e04b7a8842146294da3a96 P 1f5e4464cf2f4e57b977e56f15e323af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6ba0c4707a9480b8309f094af83bd48" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41463 } } peers { permanent_uuid: "1f5e4464cf2f4e57b977e56f15e323af" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34865 } } peers { permanent_uuid: "b44912391db4452ba9f8e77aa97ff22b" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 39043 } }
I20260811 09:08:41.128538 12280 ts_tablet_manager.cc:1428] T 23ed736026e04b7a8842146294da3a96 P e6ba0c4707a9480b8309f094af83bd48: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260811 09:08:41.130252 12276 ts_tablet_manager.cc:1428] T 23ed736026e04b7a8842146294da3a96 P 1f5e4464cf2f4e57b977e56f15e323af: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260811 09:08:41.137142 12123 raft_consensus.cc:482] T 23ed736026e04b7a8842146294da3a96 P 1f5e4464cf2f4e57b977e56f15e323af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:08:41.137380 12123 raft_consensus.cc:504] T 23ed736026e04b7a8842146294da3a96 P 1f5e4464cf2f4e57b977e56f15e323af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6ba0c4707a9480b8309f094af83bd48" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41463 } } peers { permanent_uuid: "1f5e4464cf2f4e57b977e56f15e323af" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34865 } } peers { permanent_uuid: "b44912391db4452ba9f8e77aa97ff22b" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 39043 } }
I20260811 09:08:41.138142 12123 leader_election.cc:288] T 23ed736026e04b7a8842146294da3a96 P 1f5e4464cf2f4e57b977e56f15e323af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e6ba0c4707a9480b8309f094af83bd48 (127.28.154.129:41463), b44912391db4452ba9f8e77aa97ff22b (127.28.154.131:39043)
I20260811 09:08:41.138675 11448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23ed736026e04b7a8842146294da3a96" candidate_uuid: "1f5e4464cf2f4e57b977e56f15e323af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6ba0c4707a9480b8309f094af83bd48" is_pre_election: true
I20260811 09:08:41.138959 11448 raft_consensus.cc:2454] T 23ed736026e04b7a8842146294da3a96 P e6ba0c4707a9480b8309f094af83bd48 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f5e4464cf2f4e57b977e56f15e323af in term 0.
I20260811 09:08:41.139459 11971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23ed736026e04b7a8842146294da3a96" candidate_uuid: "1f5e4464cf2f4e57b977e56f15e323af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b44912391db4452ba9f8e77aa97ff22b" is_pre_election: true
I20260811 09:08:41.139724 11971 raft_consensus.cc:2454] T 23ed736026e04b7a8842146294da3a96 P b44912391db4452ba9f8e77aa97ff22b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f5e4464cf2f4e57b977e56f15e323af in term 0.
I20260811 09:08:41.140237 11594 leader_election.cc:302] T 23ed736026e04b7a8842146294da3a96 P 1f5e4464cf2f4e57b977e56f15e323af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1f5e4464cf2f4e57b977e56f15e323af, b44912391db4452ba9f8e77aa97ff22b, e6ba0c4707a9480b8309f094af83bd48; no voters:
I20260811 09:08:41.140735 12123 raft_consensus.cc:2781] T 23ed736026e04b7a8842146294da3a96 P 1f5e4464cf2f4e57b977e56f15e323af [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 09:08:41.140846 12123 raft_consensus.cc:482] T 23ed736026e04b7a8842146294da3a96 P 1f5e4464cf2f4e57b977e56f15e323af [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 09:08:41.140946 12123 raft_consensus.cc:3037] T 23ed736026e04b7a8842146294da3a96 P 1f5e4464cf2f4e57b977e56f15e323af [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:08:41.143296 12123 raft_consensus.cc:504] T 23ed736026e04b7a8842146294da3a96 P 1f5e4464cf2f4e57b977e56f15e323af [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6ba0c4707a9480b8309f094af83bd48" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41463 } } peers { permanent_uuid: "1f5e4464cf2f4e57b977e56f15e323af" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34865 } } peers { permanent_uuid: "b44912391db4452ba9f8e77aa97ff22b" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 39043 } }
I20260811 09:08:41.144268 11448 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23ed736026e04b7a8842146294da3a96" candidate_uuid: "1f5e4464cf2f4e57b977e56f15e323af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e6ba0c4707a9480b8309f094af83bd48"
I20260811 09:08:41.144548 11448 raft_consensus.cc:3037] T 23ed736026e04b7a8842146294da3a96 P e6ba0c4707a9480b8309f094af83bd48 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:08:41.144543 11971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23ed736026e04b7a8842146294da3a96" candidate_uuid: "1f5e4464cf2f4e57b977e56f15e323af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b44912391db4452ba9f8e77aa97ff22b"
I20260811 09:08:41.144750 11971 raft_consensus.cc:3037] T 23ed736026e04b7a8842146294da3a96 P b44912391db4452ba9f8e77aa97ff22b [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:08:41.147523 11448 raft_consensus.cc:2454] T 23ed736026e04b7a8842146294da3a96 P e6ba0c4707a9480b8309f094af83bd48 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f5e4464cf2f4e57b977e56f15e323af in term 1.
I20260811 09:08:41.148042 11971 raft_consensus.cc:2454] T 23ed736026e04b7a8842146294da3a96 P b44912391db4452ba9f8e77aa97ff22b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f5e4464cf2f4e57b977e56f15e323af in term 1.
I20260811 09:08:41.148710 11598 leader_election.cc:302] T 23ed736026e04b7a8842146294da3a96 P 1f5e4464cf2f4e57b977e56f15e323af [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f5e4464cf2f4e57b977e56f15e323af, b44912391db4452ba9f8e77aa97ff22b; no voters:
I20260811 09:08:41.149088 12239 raft_consensus.cc:2781] T 23ed736026e04b7a8842146294da3a96 P 1f5e4464cf2f4e57b977e56f15e323af [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:08:41.149256 12239 raft_consensus.cc:686] T 23ed736026e04b7a8842146294da3a96 P 1f5e4464cf2f4e57b977e56f15e323af [term 1 LEADER]: Becoming Leader. State: Replica: 1f5e4464cf2f4e57b977e56f15e323af, State: Running, Role: LEADER
I20260811 09:08:41.149660 12239 consensus_queue.cc:237] T 23ed736026e04b7a8842146294da3a96 P 1f5e4464cf2f4e57b977e56f15e323af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6ba0c4707a9480b8309f094af83bd48" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41463 } } peers { permanent_uuid: "1f5e4464cf2f4e57b977e56f15e323af" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34865 } } peers { permanent_uuid: "b44912391db4452ba9f8e77aa97ff22b" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 39043 } }
I20260811 09:08:41.151356 12123 leader_election.cc:288] T 23ed736026e04b7a8842146294da3a96 P 1f5e4464cf2f4e57b977e56f15e323af [CANDIDATE]: Term 1 election: Requested vote from peers e6ba0c4707a9480b8309f094af83bd48 (127.28.154.129:41463), b44912391db4452ba9f8e77aa97ff22b (127.28.154.131:39043)
I20260811 09:08:41.153491 11162 catalog_manager.cc:5495] T 23ed736026e04b7a8842146294da3a96 P 1f5e4464cf2f4e57b977e56f15e323af reported cstate change: term changed from 0 to 1, leader changed from <none> to 1f5e4464cf2f4e57b977e56f15e323af (127.28.154.130). New cstate: current_term: 1 leader_uuid: "1f5e4464cf2f4e57b977e56f15e323af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6ba0c4707a9480b8309f094af83bd48" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1f5e4464cf2f4e57b977e56f15e323af" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34865 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b44912391db4452ba9f8e77aa97ff22b" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 39043 } health_report { overall_health: UNKNOWN } } }
I20260811 09:08:41.171692 12103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:41.166555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42770 (local address 127.28.154.130:34865)
0811 09:08:41.166739 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:41.166744 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:41.166759 (+ 15us) server_negotiation.cc:407] Connection header received
0811 09:08:41.166815 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:41.166818 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:41.166880 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:41.166961 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:08:41.167351 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:41.168149 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:41.169508 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:41.170205 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:41.170340 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:41.170899 (+ 559us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:08:41.171051 (+ 152us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:08:41.171145 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:41.171290 (+ 145us) server_negotiation.cc:299] Negotiation successful
0811 09:08:41.171378 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 09:08:41.175573 11448 raft_consensus.cc:1261] T 23ed736026e04b7a8842146294da3a96 P e6ba0c4707a9480b8309f094af83bd48 [term 1 FOLLOWER]: Refusing update from remote peer 1f5e4464cf2f4e57b977e56f15e323af: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:08:41.176258 11971 raft_consensus.cc:1261] T 23ed736026e04b7a8842146294da3a96 P b44912391db4452ba9f8e77aa97ff22b [term 1 FOLLOWER]: Refusing update from remote peer 1f5e4464cf2f4e57b977e56f15e323af: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:08:41.176383 12123 consensus_queue.cc:1035] T 23ed736026e04b7a8842146294da3a96 P 1f5e4464cf2f4e57b977e56f15e323af [LEADER]: Connected to new peer: Peer: permanent_uuid: "e6ba0c4707a9480b8309f094af83bd48" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:08:41.177047 12239 consensus_queue.cc:1035] T 23ed736026e04b7a8842146294da3a96 P 1f5e4464cf2f4e57b977e56f15e323af [LEADER]: Connected to new peer: Peer: permanent_uuid: "b44912391db4452ba9f8e77aa97ff22b" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 39043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:08:41.213349 11167 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55732:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:08:41.213668 11167 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55732:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:08:41.218603 11167 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 32c27303c7e1421abd79752385c82cd5: Sending DeleteTablet for 3 replicas of tablet 23ed736026e04b7a8842146294da3a96
I20260811 09:08:41.220147 11410 tablet_service.cc:1514] Processing DeleteTablet for tablet 23ed736026e04b7a8842146294da3a96 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:08:41 UTC) from {username='oryx', principal='oryx/127.28.154.190@KRBTEST.COM'} at 127.0.0.1:41522
I20260811 09:08:41.220525 12254 tablet_replica.cc:330] T 23ed736026e04b7a8842146294da3a96 P e6ba0c4707a9480b8309f094af83bd48: stopping tablet replica
I20260811 09:08:41.221242 11935 tablet_service.cc:1514] Processing DeleteTablet for tablet 23ed736026e04b7a8842146294da3a96 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:08:41 UTC) from {username='oryx', principal='oryx/127.28.154.190@KRBTEST.COM'} at 127.0.0.1:40632
I20260811 09:08:41.221586 12249 tablet_replica.cc:330] T 23ed736026e04b7a8842146294da3a96 P b44912391db4452ba9f8e77aa97ff22b: stopping tablet replica
I20260811 09:08:41.222383 11697 tablet_service.cc:1514] Processing DeleteTablet for tablet 23ed736026e04b7a8842146294da3a96 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:08:41 UTC) from {username='oryx', principal='oryx/127.28.154.190@KRBTEST.COM'} at 127.0.0.1:42752
I20260811 09:08:41.223253 12255 tablet_replica.cc:330] T 23ed736026e04b7a8842146294da3a96 P 1f5e4464cf2f4e57b977e56f15e323af: stopping tablet replica
I20260811 09:08:41.223510 12255 raft_consensus.cc:2229] T 23ed736026e04b7a8842146294da3a96 P 1f5e4464cf2f4e57b977e56f15e323af [term 1 LEADER]: Raft consensus shutting down.
I20260811 09:08:41.223819 12255 raft_consensus.cc:2258] T 23ed736026e04b7a8842146294da3a96 P 1f5e4464cf2f4e57b977e56f15e323af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:08:41.224840 11167 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55732:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 09:08:41.225641 11167 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 09:08:41.235710 11935 tablet_service.cc:1467] Processing CreateTablet for tablet b4ff261e1116402f81f30f52bfef1b37 (DEFAULT_TABLE table=test-table [id=4163520e5df6428d9df7f96f07742ab4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:08:41.236364 11935 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b4ff261e1116402f81f30f52bfef1b37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:08:41.236320 11410 tablet_service.cc:1467] Processing CreateTablet for tablet b4ff261e1116402f81f30f52bfef1b37 (DEFAULT_TABLE table=test-table [id=4163520e5df6428d9df7f96f07742ab4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:08:41.236932 11410 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b4ff261e1116402f81f30f52bfef1b37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:08:41.241125 11697 tablet_service.cc:1467] Processing CreateTablet for tablet b4ff261e1116402f81f30f52bfef1b37 (DEFAULT_TABLE table=test-table [id=4163520e5df6428d9df7f96f07742ab4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:08:41.241688 11697 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b4ff261e1116402f81f30f52bfef1b37. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:08:41.247396 12274 tablet_bootstrap.cc:492] T b4ff261e1116402f81f30f52bfef1b37 P b44912391db4452ba9f8e77aa97ff22b: Bootstrap starting.
I20260811 09:08:41.249482 12274 tablet_bootstrap.cc:654] T b4ff261e1116402f81f30f52bfef1b37 P b44912391db4452ba9f8e77aa97ff22b: Neither blocks nor log segments found. Creating new log.
I20260811 09:08:41.255815 12280 tablet_bootstrap.cc:492] T b4ff261e1116402f81f30f52bfef1b37 P e6ba0c4707a9480b8309f094af83bd48: Bootstrap starting.
I20260811 09:08:41.257767 12280 tablet_bootstrap.cc:654] T b4ff261e1116402f81f30f52bfef1b37 P e6ba0c4707a9480b8309f094af83bd48: Neither blocks nor log segments found. Creating new log.
I20260811 09:08:41.262694 12249 raft_consensus.cc:2229] T 23ed736026e04b7a8842146294da3a96 P b44912391db4452ba9f8e77aa97ff22b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:08:41.263489 12249 raft_consensus.cc:2258] T 23ed736026e04b7a8842146294da3a96 P b44912391db4452ba9f8e77aa97ff22b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:08:41.265934 12280 tablet_bootstrap.cc:492] T b4ff261e1116402f81f30f52bfef1b37 P e6ba0c4707a9480b8309f094af83bd48: No bootstrap required, opened a new log
I20260811 09:08:41.266146 12280 ts_tablet_manager.cc:1397] T b4ff261e1116402f81f30f52bfef1b37 P e6ba0c4707a9480b8309f094af83bd48: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.004s
I20260811 09:08:41.267002 12280 raft_consensus.cc:348] T b4ff261e1116402f81f30f52bfef1b37 P e6ba0c4707a9480b8309f094af83bd48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6ba0c4707a9480b8309f094af83bd48" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41463 } } peers { permanent_uuid: "1f5e4464cf2f4e57b977e56f15e323af" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34865 } } peers { permanent_uuid: "b44912391db4452ba9f8e77aa97ff22b" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 39043 } }
I20260811 09:08:41.267319 12280 raft_consensus.cc:374] T b4ff261e1116402f81f30f52bfef1b37 P e6ba0c4707a9480b8309f094af83bd48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:08:41.267414 12280 raft_consensus.cc:729] T b4ff261e1116402f81f30f52bfef1b37 P e6ba0c4707a9480b8309f094af83bd48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6ba0c4707a9480b8309f094af83bd48, State: Initialized, Role: FOLLOWER
I20260811 09:08:41.267704 12280 consensus_queue.cc:260] T b4ff261e1116402f81f30f52bfef1b37 P e6ba0c4707a9480b8309f094af83bd48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6ba0c4707a9480b8309f094af83bd48" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41463 } } peers { permanent_uuid: "1f5e4464cf2f4e57b977e56f15e323af" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34865 } } peers { permanent_uuid: "b44912391db4452ba9f8e77aa97ff22b" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 39043 } }
I20260811 09:08:41.268651 12280 ts_tablet_manager.cc:1428] T b4ff261e1116402f81f30f52bfef1b37 P e6ba0c4707a9480b8309f094af83bd48: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260811 09:08:41.269234 12255 ts_tablet_manager.cc:1905] T 23ed736026e04b7a8842146294da3a96 P 1f5e4464cf2f4e57b977e56f15e323af: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:08:41.276254 12276 tablet_bootstrap.cc:492] T b4ff261e1116402f81f30f52bfef1b37 P 1f5e4464cf2f4e57b977e56f15e323af: Bootstrap starting.
I20260811 09:08:41.279120 12255 ts_tablet_manager.cc:1918] T 23ed736026e04b7a8842146294da3a96 P 1f5e4464cf2f4e57b977e56f15e323af: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:08:41.283404 12276 tablet_bootstrap.cc:654] T b4ff261e1116402f81f30f52bfef1b37 P 1f5e4464cf2f4e57b977e56f15e323af: Neither blocks nor log segments found. Creating new log.
I20260811 09:08:41.283821 12255 log.cc:1196] T 23ed736026e04b7a8842146294da3a96 P 1f5e4464cf2f4e57b977e56f15e323af: Deleting WAL directory at /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439250192320-29290-0/minicluster-data/ts-1/wal/wals/23ed736026e04b7a8842146294da3a96
I20260811 09:08:41.284268 12255 ts_tablet_manager.cc:1939] T 23ed736026e04b7a8842146294da3a96 P 1f5e4464cf2f4e57b977e56f15e323af: Deleting consensus metadata
I20260811 09:08:41.285219 12274 tablet_bootstrap.cc:492] T b4ff261e1116402f81f30f52bfef1b37 P b44912391db4452ba9f8e77aa97ff22b: No bootstrap required, opened a new log
I20260811 09:08:41.285537 12274 ts_tablet_manager.cc:1397] T b4ff261e1116402f81f30f52bfef1b37 P b44912391db4452ba9f8e77aa97ff22b: Time spent bootstrapping tablet: real 0.038s user 0.017s sys 0.004s
I20260811 09:08:41.286547 12274 raft_consensus.cc:348] T b4ff261e1116402f81f30f52bfef1b37 P b44912391db4452ba9f8e77aa97ff22b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6ba0c4707a9480b8309f094af83bd48" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41463 } } peers { permanent_uuid: "1f5e4464cf2f4e57b977e56f15e323af" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34865 } } peers { permanent_uuid: "b44912391db4452ba9f8e77aa97ff22b" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 39043 } }
I20260811 09:08:41.286931 12274 raft_consensus.cc:374] T b4ff261e1116402f81f30f52bfef1b37 P b44912391db4452ba9f8e77aa97ff22b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:08:41.286931 11141 catalog_manager.cc:4841] TS 1f5e4464cf2f4e57b977e56f15e323af (127.28.154.130:34865): tablet 23ed736026e04b7a8842146294da3a96 (table test-table [id=24a0754c020045bbb1cd06bd70c08d7b]) successfully deleted
I20260811 09:08:41.287079 12274 raft_consensus.cc:729] T b4ff261e1116402f81f30f52bfef1b37 P b44912391db4452ba9f8e77aa97ff22b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b44912391db4452ba9f8e77aa97ff22b, State: Initialized, Role: FOLLOWER
I20260811 09:08:41.287386 12274 consensus_queue.cc:260] T b4ff261e1116402f81f30f52bfef1b37 P b44912391db4452ba9f8e77aa97ff22b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6ba0c4707a9480b8309f094af83bd48" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41463 } } peers { permanent_uuid: "1f5e4464cf2f4e57b977e56f15e323af" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34865 } } peers { permanent_uuid: "b44912391db4452ba9f8e77aa97ff22b" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 39043 } }
I20260811 09:08:41.289011 12274 ts_tablet_manager.cc:1428] T b4ff261e1116402f81f30f52bfef1b37 P b44912391db4452ba9f8e77aa97ff22b: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260811 09:08:41.289441 12254 raft_consensus.cc:2229] T 23ed736026e04b7a8842146294da3a96 P e6ba0c4707a9480b8309f094af83bd48 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:08:41.290016 12254 pending_rounds.cc:62] T 23ed736026e04b7a8842146294da3a96 P e6ba0c4707a9480b8309f094af83bd48: Trying to abort 2 pending ops.
I20260811 09:08:41.290227 12254 pending_rounds.cc:68] T 23ed736026e04b7a8842146294da3a96 P e6ba0c4707a9480b8309f094af83bd48: Aborting op as it isn't in flight: id { term: 1 index: 2 } timestamp: 7317255459527331840 op_type: WRITE_OP write_request { tablet_id: "23ed736026e04b7a8842146294da3a96" schema { columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } columns { name: "val" type: INT32 is_key: false is_nullable: true immutable: false } } row_operations { rows: "<redacted>""\001\003\00090\000\0001\324\000\000" indirect_data: "<redacted>""" } external_consistency_mode: CLIENT_PROPAGATED authz_token { token_data: "\010\305\317\353\323\006\"8\n\ntest-admin\022*\n 24a0754c020045bbb1cd06bd70c08d7b\020\001\030\001 \001(\001" signature: "<redacted>""Q\326\023Y&\311\2370U5\022\213\376\361\245w\275\371F\340h\222\355\314\021e\320\276V\177\314\335r4\306\035\351\"?_d\331f\271\020\314\360\261\0018\306\003\331\257.\203r\221\335\317o\331\273\277" signing_key_seq_num: 0 } } request_id { client_id: "ad36d5b4116b420d81ae0b6b30e6dc26" seq_no: 0 first_incomplete_seq_no: 0 attempt_no: 0 }
I20260811 09:08:41.291025 12254 pending_rounds.cc:68] T 23ed736026e04b7a8842146294da3a96 P e6ba0c4707a9480b8309f094af83bd48: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7317255459432361984 op_type: NO_OP noop_request { }
I20260811 09:08:41.291191 12254 raft_consensus.cc:2866] T 23ed736026e04b7a8842146294da3a96 P e6ba0c4707a9480b8309f094af83bd48 [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260811 09:08:41.291314 12254 raft_consensus.cc:2258] T 23ed736026e04b7a8842146294da3a96 P e6ba0c4707a9480b8309f094af83bd48 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:08:41.292537 12254 ts_tablet_manager.cc:1905] T 23ed736026e04b7a8842146294da3a96 P e6ba0c4707a9480b8309f094af83bd48: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:08:41.300203 12254 ts_tablet_manager.cc:1918] T 23ed736026e04b7a8842146294da3a96 P e6ba0c4707a9480b8309f094af83bd48: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:08:41.301259 12254 log.cc:1196] T 23ed736026e04b7a8842146294da3a96 P e6ba0c4707a9480b8309f094af83bd48: Deleting WAL directory at /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439250192320-29290-0/minicluster-data/ts-0/wal/wals/23ed736026e04b7a8842146294da3a96
I20260811 09:08:41.301746 12276 tablet_bootstrap.cc:492] T b4ff261e1116402f81f30f52bfef1b37 P 1f5e4464cf2f4e57b977e56f15e323af: No bootstrap required, opened a new log
I20260811 09:08:41.301805 12254 ts_tablet_manager.cc:1939] T 23ed736026e04b7a8842146294da3a96 P e6ba0c4707a9480b8309f094af83bd48: Deleting consensus metadata
I20260811 09:08:41.301852 12249 ts_tablet_manager.cc:1905] T 23ed736026e04b7a8842146294da3a96 P b44912391db4452ba9f8e77aa97ff22b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:08:41.301932 12276 ts_tablet_manager.cc:1397] T b4ff261e1116402f81f30f52bfef1b37 P 1f5e4464cf2f4e57b977e56f15e323af: Time spent bootstrapping tablet: real 0.028s user 0.000s sys 0.004s
I20260811 09:08:41.302727 12276 raft_consensus.cc:348] T b4ff261e1116402f81f30f52bfef1b37 P 1f5e4464cf2f4e57b977e56f15e323af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6ba0c4707a9480b8309f094af83bd48" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41463 } } peers { permanent_uuid: "1f5e4464cf2f4e57b977e56f15e323af" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34865 } } peers { permanent_uuid: "b44912391db4452ba9f8e77aa97ff22b" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 39043 } }
I20260811 09:08:41.302983 12276 raft_consensus.cc:374] T b4ff261e1116402f81f30f52bfef1b37 P 1f5e4464cf2f4e57b977e56f15e323af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:08:41.303052 12276 raft_consensus.cc:729] T b4ff261e1116402f81f30f52bfef1b37 P 1f5e4464cf2f4e57b977e56f15e323af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f5e4464cf2f4e57b977e56f15e323af, State: Initialized, Role: FOLLOWER
I20260811 09:08:41.303030 11141 catalog_manager.cc:4841] TS e6ba0c4707a9480b8309f094af83bd48 (127.28.154.129:41463): tablet 23ed736026e04b7a8842146294da3a96 (table test-table [id=24a0754c020045bbb1cd06bd70c08d7b]) successfully deleted
I20260811 09:08:41.303280 12276 consensus_queue.cc:260] T b4ff261e1116402f81f30f52bfef1b37 P 1f5e4464cf2f4e57b977e56f15e323af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6ba0c4707a9480b8309f094af83bd48" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41463 } } peers { permanent_uuid: "1f5e4464cf2f4e57b977e56f15e323af" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34865 } } peers { permanent_uuid: "b44912391db4452ba9f8e77aa97ff22b" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 39043 } }
I20260811 09:08:41.304091 12276 ts_tablet_manager.cc:1428] T b4ff261e1116402f81f30f52bfef1b37 P 1f5e4464cf2f4e57b977e56f15e323af: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260811 09:08:41.310828 12249 ts_tablet_manager.cc:1918] T 23ed736026e04b7a8842146294da3a96 P b44912391db4452ba9f8e77aa97ff22b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:08:41.311110 12249 log.cc:1196] T 23ed736026e04b7a8842146294da3a96 P b44912391db4452ba9f8e77aa97ff22b: Deleting WAL directory at /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439250192320-29290-0/minicluster-data/ts-2/wal/wals/23ed736026e04b7a8842146294da3a96
I20260811 09:08:41.311825 12249 ts_tablet_manager.cc:1939] T 23ed736026e04b7a8842146294da3a96 P b44912391db4452ba9f8e77aa97ff22b: Deleting consensus metadata
I20260811 09:08:41.313266 11142 catalog_manager.cc:4841] TS b44912391db4452ba9f8e77aa97ff22b (127.28.154.131:39043): tablet 23ed736026e04b7a8842146294da3a96 (table test-table [id=24a0754c020045bbb1cd06bd70c08d7b]) successfully deleted
I20260811 09:08:41.326077 12146 raft_consensus.cc:482] T b4ff261e1116402f81f30f52bfef1b37 P e6ba0c4707a9480b8309f094af83bd48 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:08:41.326318 12146 raft_consensus.cc:504] T b4ff261e1116402f81f30f52bfef1b37 P e6ba0c4707a9480b8309f094af83bd48 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6ba0c4707a9480b8309f094af83bd48" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41463 } } peers { permanent_uuid: "1f5e4464cf2f4e57b977e56f15e323af" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34865 } } peers { permanent_uuid: "b44912391db4452ba9f8e77aa97ff22b" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 39043 } }
I20260811 09:08:41.327116 12146 leader_election.cc:288] T b4ff261e1116402f81f30f52bfef1b37 P e6ba0c4707a9480b8309f094af83bd48 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f5e4464cf2f4e57b977e56f15e323af (127.28.154.130:34865), b44912391db4452ba9f8e77aa97ff22b (127.28.154.131:39043)
I20260811 09:08:41.328055 11737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4ff261e1116402f81f30f52bfef1b37" candidate_uuid: "e6ba0c4707a9480b8309f094af83bd48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f5e4464cf2f4e57b977e56f15e323af" is_pre_election: true
I20260811 09:08:41.328892 11737 raft_consensus.cc:2454] T b4ff261e1116402f81f30f52bfef1b37 P 1f5e4464cf2f4e57b977e56f15e323af [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6ba0c4707a9480b8309f094af83bd48 in term 0.
I20260811 09:08:41.329504 11323 leader_election.cc:302] T b4ff261e1116402f81f30f52bfef1b37 P e6ba0c4707a9480b8309f094af83bd48 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f5e4464cf2f4e57b977e56f15e323af, e6ba0c4707a9480b8309f094af83bd48; no voters:
I20260811 09:08:41.329890 12146 raft_consensus.cc:2781] T b4ff261e1116402f81f30f52bfef1b37 P e6ba0c4707a9480b8309f094af83bd48 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 09:08:41.330006 12146 raft_consensus.cc:482] T b4ff261e1116402f81f30f52bfef1b37 P e6ba0c4707a9480b8309f094af83bd48 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 09:08:41.330091 12146 raft_consensus.cc:3037] T b4ff261e1116402f81f30f52bfef1b37 P e6ba0c4707a9480b8309f094af83bd48 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:08:41.333261 12146 raft_consensus.cc:504] T b4ff261e1116402f81f30f52bfef1b37 P e6ba0c4707a9480b8309f094af83bd48 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6ba0c4707a9480b8309f094af83bd48" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41463 } } peers { permanent_uuid: "1f5e4464cf2f4e57b977e56f15e323af" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34865 } } peers { permanent_uuid: "b44912391db4452ba9f8e77aa97ff22b" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 39043 } }
I20260811 09:08:41.334493 11737 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4ff261e1116402f81f30f52bfef1b37" candidate_uuid: "e6ba0c4707a9480b8309f094af83bd48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f5e4464cf2f4e57b977e56f15e323af"
I20260811 09:08:41.334744 11737 raft_consensus.cc:3037] T b4ff261e1116402f81f30f52bfef1b37 P 1f5e4464cf2f4e57b977e56f15e323af [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:08:41.338404 11973 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4ff261e1116402f81f30f52bfef1b37" candidate_uuid: "e6ba0c4707a9480b8309f094af83bd48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b44912391db4452ba9f8e77aa97ff22b" is_pre_election: true
I20260811 09:08:41.338758 11973 raft_consensus.cc:2454] T b4ff261e1116402f81f30f52bfef1b37 P b44912391db4452ba9f8e77aa97ff22b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6ba0c4707a9480b8309f094af83bd48 in term 0.
I20260811 09:08:41.340018 11737 raft_consensus.cc:2454] T b4ff261e1116402f81f30f52bfef1b37 P 1f5e4464cf2f4e57b977e56f15e323af [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6ba0c4707a9480b8309f094af83bd48 in term 1.
I20260811 09:08:41.340641 11323 leader_election.cc:302] T b4ff261e1116402f81f30f52bfef1b37 P e6ba0c4707a9480b8309f094af83bd48 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f5e4464cf2f4e57b977e56f15e323af, e6ba0c4707a9480b8309f094af83bd48; no voters:
I20260811 09:08:41.343747 11973 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4ff261e1116402f81f30f52bfef1b37" candidate_uuid: "e6ba0c4707a9480b8309f094af83bd48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b44912391db4452ba9f8e77aa97ff22b"
I20260811 09:08:41.344038 11973 raft_consensus.cc:3037] T b4ff261e1116402f81f30f52bfef1b37 P b44912391db4452ba9f8e77aa97ff22b [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:08:41.344689 12298 raft_consensus.cc:2781] T b4ff261e1116402f81f30f52bfef1b37 P e6ba0c4707a9480b8309f094af83bd48 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:08:41.347646 12146 leader_election.cc:288] T b4ff261e1116402f81f30f52bfef1b37 P e6ba0c4707a9480b8309f094af83bd48 [CANDIDATE]: Term 1 election: Requested vote from peers 1f5e4464cf2f4e57b977e56f15e323af (127.28.154.130:34865), b44912391db4452ba9f8e77aa97ff22b (127.28.154.131:39043)
I20260811 09:08:41.350232 11973 raft_consensus.cc:2454] T b4ff261e1116402f81f30f52bfef1b37 P b44912391db4452ba9f8e77aa97ff22b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6ba0c4707a9480b8309f094af83bd48 in term 1.
I20260811 09:08:41.351259 12298 raft_consensus.cc:686] T b4ff261e1116402f81f30f52bfef1b37 P e6ba0c4707a9480b8309f094af83bd48 [term 1 LEADER]: Becoming Leader. State: Replica: e6ba0c4707a9480b8309f094af83bd48, State: Running, Role: LEADER
I20260811 09:08:41.351683 12298 consensus_queue.cc:237] T b4ff261e1116402f81f30f52bfef1b37 P e6ba0c4707a9480b8309f094af83bd48 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6ba0c4707a9480b8309f094af83bd48" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41463 } } peers { permanent_uuid: "1f5e4464cf2f4e57b977e56f15e323af" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34865 } } peers { permanent_uuid: "b44912391db4452ba9f8e77aa97ff22b" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 39043 } }
I20260811 09:08:41.365213 11167 catalog_manager.cc:5495] T b4ff261e1116402f81f30f52bfef1b37 P e6ba0c4707a9480b8309f094af83bd48 reported cstate change: term changed from 0 to 1, leader changed from <none> to e6ba0c4707a9480b8309f094af83bd48 (127.28.154.129). New cstate: current_term: 1 leader_uuid: "e6ba0c4707a9480b8309f094af83bd48" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6ba0c4707a9480b8309f094af83bd48" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1f5e4464cf2f4e57b977e56f15e323af" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34865 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b44912391db4452ba9f8e77aa97ff22b" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 39043 } health_report { overall_health: UNKNOWN } } }
I20260811 09:08:41.410827 12303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:41.403099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41544 (local address 127.28.154.129:41463)
0811 09:08:41.403650 (+ 551us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:41.403657 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:41.403684 (+ 27us) server_negotiation.cc:407] Connection header received
0811 09:08:41.403753 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:41.403759 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:41.403844 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:41.404482 (+ 638us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:08:41.404691 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:41.405900 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:41.407942 (+ 2042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:41.408637 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:41.409277 (+ 640us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:41.409387 (+ 110us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:08:41.409523 (+ 136us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:08:41.410257 (+ 734us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:41.410312 (+ 55us) server_negotiation.cc:299] Negotiation successful
0811 09:08:41.410400 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":407,"thread_start_us":268,"threads_started":1}
I20260811 09:08:41.451488 12103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:41.433415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.129:40605 (local address 127.28.154.130:34865)
0811 09:08:41.433631 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:41.433637 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:41.444106 (+ 10469us) server_negotiation.cc:407] Connection header received
0811 09:08:41.444796 (+ 690us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:41.444803 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:41.444901 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:41.444973 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:08:41.445804 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:41.446958 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:41.449145 (+ 2187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:41.449811 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:41.449919 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:41.450623 (+ 704us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:08:41.450762 (+ 139us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:08:41.450837 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:41.451064 (+ 227us) server_negotiation.cc:299] Negotiation successful
0811 09:08:41.451166 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260811 09:08:41.452612 12307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:41.433288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.130:34865 (local address 127.28.154.129:40605)
0811 09:08:41.443934 (+ 10646us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:41.443962 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:41.444591 (+ 629us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:41.445215 (+ 624us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:41.445220 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:41.445234 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 09:08:41.445598 (+ 364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:41.445615 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:41.447164 (+ 1549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:41.447169 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:41.448969 (+ 1800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:41.448987 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:41.450119 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:41.450125 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:41.450310 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:41.450390 (+ 80us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 09:08:41.450923 (+ 533us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 09:08:41.450933 (+ 10us) client_negotiation.cc:769] Sending connection context
0811 09:08:41.452009 (+ 1076us) client_negotiation.cc:240] Negotiation successful
0811 09:08:41.452165 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10391,"thread_start_us":10848,"threads_started":1}
I20260811 09:08:41.469465 11973 raft_consensus.cc:1261] T b4ff261e1116402f81f30f52bfef1b37 P b44912391db4452ba9f8e77aa97ff22b [term 1 FOLLOWER]: Refusing update from remote peer e6ba0c4707a9480b8309f094af83bd48: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:08:41.469467 11737 raft_consensus.cc:1261] T b4ff261e1116402f81f30f52bfef1b37 P 1f5e4464cf2f4e57b977e56f15e323af [term 1 FOLLOWER]: Refusing update from remote peer e6ba0c4707a9480b8309f094af83bd48: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:08:41.470182 12298 consensus_queue.cc:1035] T b4ff261e1116402f81f30f52bfef1b37 P e6ba0c4707a9480b8309f094af83bd48 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b44912391db4452ba9f8e77aa97ff22b" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 39043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:08:41.470522 12300 consensus_queue.cc:1035] T b4ff261e1116402f81f30f52bfef1b37 P e6ba0c4707a9480b8309f094af83bd48 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f5e4464cf2f4e57b977e56f15e323af" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34865 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:08:41.552363 12208 mvcc.cc:205] Tried to move back new op lower bound from 7317255460732383232 to 7317255460260020224. Current Snapshot: MvccSnapshot[applied={T|T < 7317255460732383232}]
I20260811 09:08:41.553598 12303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:41.545387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.130:45635 (local address 127.28.154.129:41463)
0811 09:08:41.545596 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:41.545602 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:41.546794 (+ 1192us) server_negotiation.cc:407] Connection header received
0811 09:08:41.546975 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:41.546981 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:41.547100 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:41.547185 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:08:41.548025 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:41.549185 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:41.551453 (+ 2268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:41.552056 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:41.552154 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:41.552782 (+ 628us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:08:41.552928 (+ 146us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:08:41.552989 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:41.553232 (+ 243us) server_negotiation.cc:299] Negotiation successful
0811 09:08:41.553322 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 09:08:41.554680 12312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:41.544933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.129:41463 (local address 127.28.154.130:45635)
0811 09:08:41.546674 (+ 1741us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:41.546691 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:41.546836 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:41.547432 (+ 596us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:41.547437 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:41.547450 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 09:08:41.547790 (+ 340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:41.547803 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:41.549579 (+ 1776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:41.549584 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:41.551280 (+ 1696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:41.551296 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:41.552365 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:41.552372 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:41.552545 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:41.552601 (+ 56us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 09:08:41.553078 (+ 477us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 09:08:41.553088 (+ 10us) client_negotiation.cc:769] Sending connection context
0811 09:08:41.554072 (+ 984us) client_negotiation.cc:240] Negotiation successful
0811 09:08:41.554226 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1575,"thread_start_us":1423,"threads_started":1}
I20260811 09:08:41.595582 11167 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55732:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:08:41.595860 11167 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55732:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:08:41.598974 11167 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 32c27303c7e1421abd79752385c82cd5: Sending DeleteTablet for 3 replicas of tablet b4ff261e1116402f81f30f52bfef1b37
I20260811 09:08:41.599822 11410 tablet_service.cc:1514] Processing DeleteTablet for tablet b4ff261e1116402f81f30f52bfef1b37 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:08:41 UTC) from {username='oryx', principal='oryx/127.28.154.190@KRBTEST.COM'} at 127.0.0.1:41522
I20260811 09:08:41.599893 11935 tablet_service.cc:1514] Processing DeleteTablet for tablet b4ff261e1116402f81f30f52bfef1b37 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:08:41 UTC) from {username='oryx', principal='oryx/127.28.154.190@KRBTEST.COM'} at 127.0.0.1:40632
I20260811 09:08:41.599922 11697 tablet_service.cc:1514] Processing DeleteTablet for tablet b4ff261e1116402f81f30f52bfef1b37 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:08:41 UTC) from {username='oryx', principal='oryx/127.28.154.190@KRBTEST.COM'} at 127.0.0.1:42752
I20260811 09:08:41.600167 12249 tablet_replica.cc:330] T b4ff261e1116402f81f30f52bfef1b37 P b44912391db4452ba9f8e77aa97ff22b: stopping tablet replica
I20260811 09:08:41.600160 12254 tablet_replica.cc:330] T b4ff261e1116402f81f30f52bfef1b37 P e6ba0c4707a9480b8309f094af83bd48: stopping tablet replica
I20260811 09:08:41.600265 12255 tablet_replica.cc:330] T b4ff261e1116402f81f30f52bfef1b37 P 1f5e4464cf2f4e57b977e56f15e323af: stopping tablet replica
I20260811 09:08:41.600360 12254 raft_consensus.cc:2229] T b4ff261e1116402f81f30f52bfef1b37 P e6ba0c4707a9480b8309f094af83bd48 [term 1 LEADER]: Raft consensus shutting down.
I20260811 09:08:41.600360 12249 raft_consensus.cc:2229] T b4ff261e1116402f81f30f52bfef1b37 P b44912391db4452ba9f8e77aa97ff22b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:08:41.600488 12255 raft_consensus.cc:2229] T b4ff261e1116402f81f30f52bfef1b37 P 1f5e4464cf2f4e57b977e56f15e323af [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:08:41.600585 12249 raft_consensus.cc:2258] T b4ff261e1116402f81f30f52bfef1b37 P b44912391db4452ba9f8e77aa97ff22b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:08:41.600718 12254 raft_consensus.cc:2258] T b4ff261e1116402f81f30f52bfef1b37 P e6ba0c4707a9480b8309f094af83bd48 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:08:41.600730 12255 raft_consensus.cc:2258] T b4ff261e1116402f81f30f52bfef1b37 P 1f5e4464cf2f4e57b977e56f15e323af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:08:41.601958 12249 ts_tablet_manager.cc:1905] T b4ff261e1116402f81f30f52bfef1b37 P b44912391db4452ba9f8e77aa97ff22b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:08:41.602667 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 11294
I20260811 09:08:41.602748 12255 ts_tablet_manager.cc:1905] T b4ff261e1116402f81f30f52bfef1b37 P 1f5e4464cf2f4e57b977e56f15e323af: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:08:41.608851 12254 ts_tablet_manager.cc:1905] T b4ff261e1116402f81f30f52bfef1b37 P e6ba0c4707a9480b8309f094af83bd48: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:08:41.609479 12249 ts_tablet_manager.cc:1918] T b4ff261e1116402f81f30f52bfef1b37 P b44912391db4452ba9f8e77aa97ff22b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 09:08:41.609717 12249 log.cc:1196] T b4ff261e1116402f81f30f52bfef1b37 P b44912391db4452ba9f8e77aa97ff22b: Deleting WAL directory at /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439250192320-29290-0/minicluster-data/ts-2/wal/wals/b4ff261e1116402f81f30f52bfef1b37
I20260811 09:08:41.610168 12249 ts_tablet_manager.cc:1939] T b4ff261e1116402f81f30f52bfef1b37 P b44912391db4452ba9f8e77aa97ff22b: Deleting consensus metadata
I20260811 09:08:41.611629 11142 catalog_manager.cc:4841] TS b44912391db4452ba9f8e77aa97ff22b (127.28.154.131:39043): tablet b4ff261e1116402f81f30f52bfef1b37 (table test-table [id=4163520e5df6428d9df7f96f07742ab4]) successfully deleted
I20260811 09:08:41.616150 12255 ts_tablet_manager.cc:1918] T b4ff261e1116402f81f30f52bfef1b37 P 1f5e4464cf2f4e57b977e56f15e323af: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 09:08:41.616318 12254 ts_tablet_manager.cc:1918] T b4ff261e1116402f81f30f52bfef1b37 P e6ba0c4707a9480b8309f094af83bd48: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 09:08:41.616523 12254 log.cc:1196] T b4ff261e1116402f81f30f52bfef1b37 P e6ba0c4707a9480b8309f094af83bd48: Deleting WAL directory at /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439250192320-29290-0/minicluster-data/ts-0/wal/wals/b4ff261e1116402f81f30f52bfef1b37
I20260811 09:08:41.616508 12255 log.cc:1196] T b4ff261e1116402f81f30f52bfef1b37 P 1f5e4464cf2f4e57b977e56f15e323af: Deleting WAL directory at /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786439250192320-29290-0/minicluster-data/ts-1/wal/wals/b4ff261e1116402f81f30f52bfef1b37
I20260811 09:08:41.617027 12254 ts_tablet_manager.cc:1939] T b4ff261e1116402f81f30f52bfef1b37 P e6ba0c4707a9480b8309f094af83bd48: Deleting consensus metadata
I20260811 09:08:41.617034 12255 ts_tablet_manager.cc:1939] T b4ff261e1116402f81f30f52bfef1b37 P 1f5e4464cf2f4e57b977e56f15e323af: Deleting consensus metadata
I20260811 09:08:41.618398 11141 catalog_manager.cc:4841] TS 1f5e4464cf2f4e57b977e56f15e323af (127.28.154.130:34865): tablet b4ff261e1116402f81f30f52bfef1b37 (table test-table [id=4163520e5df6428d9df7f96f07742ab4]) successfully deleted
I20260811 09:08:41.618673 11141 catalog_manager.cc:4841] TS e6ba0c4707a9480b8309f094af83bd48 (127.28.154.129:41463): tablet b4ff261e1116402f81f30f52bfef1b37 (table test-table [id=4163520e5df6428d9df7f96f07742ab4]) successfully deleted
Aug 11 09:08:41 dist-test-slave-xt4k krb5kdc[11078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439320, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.28.154.129@KRBTEST.COM
I20260811 09:08:41.630050 12303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:41.607935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41560 (local address 127.28.154.129:41463)
0811 09:08:41.608153 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:41.608159 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:41.612511 (+ 4352us) server_negotiation.cc:407] Connection header received
0811 09:08:41.612798 (+ 287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:41.612805 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:41.612951 (+ 146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:41.613065 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:41.614974 (+ 1909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:41.616083 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:41.618008 (+ 1925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:41.618675 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:41.618790 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:41.623024 (+ 4234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:41.623045 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:41.623071 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:08:41.623126 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:41.627030 (+ 3904us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:41.627838 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:41.627846 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:41.627849 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:41.628014 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:41.628420 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:41.628424 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:41.628426 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:41.628831 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:41.629163 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:41.629469 (+ 306us) server_negotiation.cc:299] Negotiation successful
0811 09:08:41.629696 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 09:08:41.630716 11541 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:08:41.759907 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 11294
I20260811 09:08:41.800467 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 11559
Aug 11 09:08:41 dist-test-slave-xt4k krb5kdc[11078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439320, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.28.154.130@KRBTEST.COM
I20260811 09:08:41.816064 12103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:41.801344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42780 (local address 127.28.154.130:34865)
0811 09:08:41.801537 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:41.801543 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:41.801563 (+ 20us) server_negotiation.cc:407] Connection header received
0811 09:08:41.801718 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:41.801737 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:41.801826 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:41.801953 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:41.803187 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:41.803911 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:41.805414 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:41.805820 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:41.805936 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:41.810138 (+ 4202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:41.810153 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:41.810171 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:08:41.810207 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:41.813210 (+ 3003us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:41.813898 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:41.813906 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:41.813908 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:41.814040 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:41.814510 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:41.814515 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:41.814517 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:41.814909 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:41.815180 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:41.815526 (+ 346us) server_negotiation.cc:299] Negotiation successful
0811 09:08:41.815752 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 09:08:41.816823 11809 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:08:41.965078 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 11559
I20260811 09:08:41.998207 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 11824
Aug 11 09:08:42 dist-test-slave-xt4k krb5kdc[11078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439320, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.28.154.131@KRBTEST.COM
I20260811 09:08:42.014029 12317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:41.998957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40654 (local address 127.28.154.131:39043)
0811 09:08:41.999415 (+ 458us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:41.999420 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:41.999437 (+ 17us) server_negotiation.cc:407] Connection header received
0811 09:08:41.999491 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:41.999495 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:41.999556 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:41.999656 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:42.001057 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:42.001795 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:42.003117 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:42.003560 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:42.003680 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:42.008160 (+ 4480us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:42.008177 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:42.008196 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:08:42.008242 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:42.011017 (+ 2775us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:42.011758 (+ 741us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:42.011766 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:42.011769 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:42.011918 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:42.012446 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:42.012451 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:42.012454 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:42.012914 (+ 460us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:42.013227 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:42.013544 (+ 317us) server_negotiation.cc:299] Negotiation successful
0811 09:08:42.013754 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":245,"threads_started":1}
I20260811 09:08:42.014719 12049 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:08:42.138836 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 11824
I20260811 09:08:42.169121 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 11099
I20260811 09:08:42.170408 11241 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:08:42.290158 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 11099
2026-08-11T09:08:42Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (6731 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786439250192320-29290-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:08:42 dist-test-slave-xt4k krb5kdc[12325](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:08:42 dist-test-slave-xt4k krb5kdc[12325](info): set up 2 sockets
Aug 11 09:08:42 dist-test-slave-xt4k krb5kdc[12325](info): commencing operation
krb5kdc: starting...
W20260811 09:08:43.696159 29290 mini_kdc.cc:121] Time spent starting KDC: real 1.349s user 0.000s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786439250192320-29290-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786439250192320-29290-0/krb5kdc/test-user.keytab.
Aug 11 09:08:43 dist-test-slave-xt4k krb5kdc[12325](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439323, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:08:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:08:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.154.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.154.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.154.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab.
Entry for principal kudu/127.28.154.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab.
I20260811 09:08:43.993311 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786439250192320-29290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786439250192320-29290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786439250192320-29290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.154.190:43693
--webserver_interface=127.28.154.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab
--principal=kudu/127.28.154.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:32845
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.154.190:43693
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:08:44.343931 12351 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:08:44.344262 12351 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:08:44.344413 12351 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:08:44.351886 12351 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:08:44.352005 12351 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:08:44.352059 12351 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:08:44.352142 12351 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:08:44.352206 12351 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:08:44.362407 12351 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:32845
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786439250192320-29290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786439250192320-29290-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.154.190:43693
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.154.190:43693
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.154.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786439250192320-29290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:08:44.363761 12351 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:08:44.365156 12351 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:08:44.377890 12358 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:08:44.377923 12365 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:08:44.378767 12357 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:08:44.378805 12351 server_base.cc:1034] running on GCE node
I20260811 09:08:44.379659 12351 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:08:44.381213 12351 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:08:44.382439 12351 hybrid_clock.cc:648] HybridClock initialized: now 1786439324382382 us; error 49 us; skew 500 ppm
I20260811 09:08:44.382803 12351 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-11T09:08:44Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (2155 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:08:44 dist-test-slave-xt4k krb5kdc[12376](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:08:44 dist-test-slave-xt4k krb5kdc[12376](info): set up 2 sockets
Aug 11 09:08:44 dist-test-slave-xt4k krb5kdc[12376](info): commencing operation
krb5kdc: starting...
W20260811 09:08:45.766040 29290 mini_kdc.cc:121] Time spent starting KDC: real 1.263s user 0.000s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/krb5kdc/test-user.keytab.
Aug 11 09:08:45 dist-test-slave-xt4k krb5kdc[12376](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439325, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:08:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:08:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.154.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.154.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.154.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab.
Entry for principal kudu/127.28.154.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab.
I20260811 09:08:46.052068 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.154.190:39729
--webserver_interface=127.28.154.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab
--principal=kudu/127.28.154.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:45223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.154.190:39729
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:08:46.401257 12397 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:08:46.401583 12397 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:08:46.401723 12397 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:08:46.409267 12397 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:08:46.409415 12397 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:08:46.409509 12397 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:08:46.409592 12397 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:08:46.409654 12397 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:08:46.420051 12397 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:45223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.154.190:39729
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab
--principal=kudu/127.28.154.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.154.190:39729
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.154.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:08:46.421552 12397 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:08:46.422919 12397 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:08:46.435220 12404 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:08:46.435343 12403 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:08:46.435346 12407 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:08:46.436530 12397 server_base.cc:1034] running on GCE node
I20260811 09:08:46.437438 12397 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:08:46.439036 12397 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:08:46.440292 12397 hybrid_clock.cc:648] HybridClock initialized: now 1786439326440236 us; error 54 us; skew 500 ppm
I20260811 09:08:46.440757 12397 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:08:46 dist-test-slave-xt4k krb5kdc[12376](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439326, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:08:46.448733 12397 init.cc:377] Logged in from keytab as kudu/127.28.154.190@KRBTEST.COM (short username kudu)
I20260811 09:08:46.450448 12397 webserver.cc:460] Webserver started at http://127.28.154.190:39147/ using document root <none> and password file <none>
I20260811 09:08:46.451207 12397 fs_manager.cc:359] Metadata directory not provided
I20260811 09:08:46.451332 12397 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:08:46.451654 12397 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:08:46.455000 12397 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/master-0/data/instance:
uuid: "286c0ebf7b5d4e8f9bfd87bf58846b74"
format_stamp: "Formatted at 2026-08-11 09:08:46 on dist-test-slave-xt4k"
server_key: "4b4fdf45fee7bd41e24a8a20bfdf569a"
server_key_iv: "90a041e128e317b0e2efd7a29f655baa"
server_key_version: "encryptionkey@0"
I20260811 09:08:46.455788 12397 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/master-0/wal/instance:
uuid: "286c0ebf7b5d4e8f9bfd87bf58846b74"
format_stamp: "Formatted at 2026-08-11 09:08:46 on dist-test-slave-xt4k"
server_key: "4b4fdf45fee7bd41e24a8a20bfdf569a"
server_key_iv: "90a041e128e317b0e2efd7a29f655baa"
server_key_version: "encryptionkey@0"
I20260811 09:08:46.461297 12397 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260811 09:08:46.465561 12419 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:08:46.467360 12397 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.004s
I20260811 09:08:46.467571 12397 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/master-0/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/master-0/wal
uuid: "286c0ebf7b5d4e8f9bfd87bf58846b74"
format_stamp: "Formatted at 2026-08-11 09:08:46 on dist-test-slave-xt4k"
server_key: "4b4fdf45fee7bd41e24a8a20bfdf569a"
server_key_iv: "90a041e128e317b0e2efd7a29f655baa"
server_key_version: "encryptionkey@0"
I20260811 09:08:46.467789 12397 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:08:46.489105 12397 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:08:46.494390 12397 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:08:46.494694 12397 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:08:46.513298 12397 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.190:39729
I20260811 09:08:46.513321 12501 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.190:39729 every 8 connection(s)
I20260811 09:08:46.514993 12397 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
I20260811 09:08:46.519217 12503 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:08:46.519960 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 12397
I20260811 09:08:46.520169 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/master-0/wal/instance
I20260811 09:08:46.520872 29290 external_mini_cluster.cc:1428] Setting key 6165f56fd4cd976bc860a00a95f57cb0
I20260811 09:08:46.533639 12503 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 286c0ebf7b5d4e8f9bfd87bf58846b74: Bootstrap starting.
Aug 11 09:08:46 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439325, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.154.190@KRBTEST.COM
I20260811 09:08:46.537623 12503 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 286c0ebf7b5d4e8f9bfd87bf58846b74: Neither blocks nor log segments found. Creating new log.
I20260811 09:08:46.538851 12503 log.cc:824] T 00000000000000000000000000000000 P 286c0ebf7b5d4e8f9bfd87bf58846b74: Log is configured to *not* fsync() on all Append() calls
I20260811 09:08:46.542272 12503 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 286c0ebf7b5d4e8f9bfd87bf58846b74: No bootstrap required, opened a new log
I20260811 09:08:46.545228 12508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:46.523121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56482 (local address 127.28.154.190:39729)
0811 09:08:46.523926 (+ 805us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:46.523938 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:46.523982 (+ 44us) server_negotiation.cc:407] Connection header received
0811 09:08:46.525667 (+ 1685us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:46.525697 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:46.526123 (+ 426us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:46.526481 (+ 358us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:46.528292 (+ 1811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:46.529692 (+ 1400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:46.531512 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:46.532221 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:46.532392 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:46.537321 (+ 4929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:46.537331 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:46.537347 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:08:46.537389 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:46.540940 (+ 3551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:46.541813 (+ 873us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:46.541820 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:46.541826 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:46.541987 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:46.542415 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:46.542420 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:46.542423 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:46.542984 (+ 561us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:46.543219 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:46.543685 (+ 466us) server_negotiation.cc:299] Negotiation successful
0811 09:08:46.544041 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":580,"thread_start_us":348,"threads_started":1}
I20260811 09:08:46.551471 12503 raft_consensus.cc:348] T 00000000000000000000000000000000 P 286c0ebf7b5d4e8f9bfd87bf58846b74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "286c0ebf7b5d4e8f9bfd87bf58846b74" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 39729 } }
I20260811 09:08:46.551867 12503 raft_consensus.cc:374] T 00000000000000000000000000000000 P 286c0ebf7b5d4e8f9bfd87bf58846b74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:08:46.551971 12503 raft_consensus.cc:729] T 00000000000000000000000000000000 P 286c0ebf7b5d4e8f9bfd87bf58846b74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 286c0ebf7b5d4e8f9bfd87bf58846b74, State: Initialized, Role: FOLLOWER
I20260811 09:08:46.552577 12503 consensus_queue.cc:260] T 00000000000000000000000000000000 P 286c0ebf7b5d4e8f9bfd87bf58846b74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "286c0ebf7b5d4e8f9bfd87bf58846b74" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 39729 } }
I20260811 09:08:46.552799 12503 raft_consensus.cc:388] T 00000000000000000000000000000000 P 286c0ebf7b5d4e8f9bfd87bf58846b74 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:08:46.552932 12503 raft_consensus.cc:482] T 00000000000000000000000000000000 P 286c0ebf7b5d4e8f9bfd87bf58846b74 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:08:46.553131 12503 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 286c0ebf7b5d4e8f9bfd87bf58846b74 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:08:46.555533 12503 raft_consensus.cc:504] T 00000000000000000000000000000000 P 286c0ebf7b5d4e8f9bfd87bf58846b74 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "286c0ebf7b5d4e8f9bfd87bf58846b74" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 39729 } }
I20260811 09:08:46.556166 12503 leader_election.cc:302] T 00000000000000000000000000000000 P 286c0ebf7b5d4e8f9bfd87bf58846b74 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 286c0ebf7b5d4e8f9bfd87bf58846b74; no voters:
I20260811 09:08:46.556831 12503 leader_election.cc:288] T 00000000000000000000000000000000 P 286c0ebf7b5d4e8f9bfd87bf58846b74 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:08:46.556972 12511 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 286c0ebf7b5d4e8f9bfd87bf58846b74 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:08:46.557665 12511 raft_consensus.cc:686] T 00000000000000000000000000000000 P 286c0ebf7b5d4e8f9bfd87bf58846b74 [term 1 LEADER]: Becoming Leader. State: Replica: 286c0ebf7b5d4e8f9bfd87bf58846b74, State: Running, Role: LEADER
I20260811 09:08:46.558195 12511 consensus_queue.cc:237] T 00000000000000000000000000000000 P 286c0ebf7b5d4e8f9bfd87bf58846b74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "286c0ebf7b5d4e8f9bfd87bf58846b74" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 39729 } }
I20260811 09:08:46.558846 12503 sys_catalog.cc:564] T 00000000000000000000000000000000 P 286c0ebf7b5d4e8f9bfd87bf58846b74 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:08:46.561117 12512 sys_catalog.cc:455] T 00000000000000000000000000000000 P 286c0ebf7b5d4e8f9bfd87bf58846b74 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "286c0ebf7b5d4e8f9bfd87bf58846b74" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "286c0ebf7b5d4e8f9bfd87bf58846b74" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 39729 } } }
I20260811 09:08:46.561645 12512 sys_catalog.cc:458] T 00000000000000000000000000000000 P 286c0ebf7b5d4e8f9bfd87bf58846b74 [sys.catalog]: This master's current role is: LEADER
I20260811 09:08:46.561558 12514 sys_catalog.cc:455] T 00000000000000000000000000000000 P 286c0ebf7b5d4e8f9bfd87bf58846b74 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 286c0ebf7b5d4e8f9bfd87bf58846b74. Latest consensus state: current_term: 1 leader_uuid: "286c0ebf7b5d4e8f9bfd87bf58846b74" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "286c0ebf7b5d4e8f9bfd87bf58846b74" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 39729 } } }
I20260811 09:08:46.561864 12514 sys_catalog.cc:458] T 00000000000000000000000000000000 P 286c0ebf7b5d4e8f9bfd87bf58846b74 [sys.catalog]: This master's current role is: LEADER
I20260811 09:08:46.564409 12524 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:08:46.570444 12524 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:08:46.579666 12524 catalog_manager.cc:1347] Generated new cluster ID: 8a2d35833a964c3f92e20cb489e342bd
I20260811 09:08:46.579800 12524 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:08:46.631691 12524 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:08:46.633147 12524 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:08:46.641274 12524 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 286c0ebf7b5d4e8f9bfd87bf58846b74: Generated new TSK 0
I20260811 09:08:46.642181 12524 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.28.154.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.154.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.154.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.129.keytab.
Entry for principal kudu/127.28.154.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.129.keytab.
I20260811 09:08:46.707517 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.154.129:0
--local_ip_for_outbound_sockets=127.28.154.129
--webserver_interface=127.28.154.129
--webserver_port=0
--tserver_master_addrs=127.28.154.190:39729
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.129.keytab
--principal=kudu/127.28.154.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:45223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:08:47.052608 12550 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:08:47.052881 12550 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:08:47.053007 12550 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:08:47.060285 12550 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:08:47.060412 12550 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:08:47.060560 12550 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.154.129
I20260811 09:08:47.070616 12550 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:45223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.129.keytab
--principal=kudu/127.28.154.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.154.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.154.129
--webserver_port=0
--tserver_master_addrs=127.28.154.190:39729
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.154.129
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:08:47.071986 12550 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:08:47.073505 12550 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:08:47.086410 12556 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:08:47.086329 12557 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:08:47.086416 12560 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:08:47.087270 12550 server_base.cc:1034] running on GCE node
I20260811 09:08:47.087847 12550 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:08:47.088858 12550 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:08:47.090088 12550 hybrid_clock.cc:648] HybridClock initialized: now 1786439327090046 us; error 36 us; skew 500 ppm
I20260811 09:08:47.090426 12550 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:08:47 dist-test-slave-xt4k krb5kdc[12376](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439327, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:08:47.093747 12550 init.cc:377] Logged in from keytab as kudu/127.28.154.129@KRBTEST.COM (short username kudu)
I20260811 09:08:47.095289 12550 webserver.cc:460] Webserver started at http://127.28.154.129:39655/ using document root <none> and password file <none>
I20260811 09:08:47.095979 12550 fs_manager.cc:359] Metadata directory not provided
I20260811 09:08:47.096095 12550 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:08:47.096436 12550 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:08:47.099392 12550 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/ts-0/data/instance:
uuid: "cb106c4f38054c378ba28afa9d3b6054"
format_stamp: "Formatted at 2026-08-11 09:08:47 on dist-test-slave-xt4k"
server_key: "9a4d66f898c271031744050f0657e0dc"
server_key_iv: "89e63e2a6c37fdc27dd400d6d19e0766"
server_key_version: "encryptionkey@0"
I20260811 09:08:47.100236 12550 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/ts-0/wal/instance:
uuid: "cb106c4f38054c378ba28afa9d3b6054"
format_stamp: "Formatted at 2026-08-11 09:08:47 on dist-test-slave-xt4k"
server_key: "9a4d66f898c271031744050f0657e0dc"
server_key_iv: "89e63e2a6c37fdc27dd400d6d19e0766"
server_key_version: "encryptionkey@0"
I20260811 09:08:47.105682 12550 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260811 09:08:47.109354 12577 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:08:47.110824 12550 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 09:08:47.111069 12550 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/ts-0/wal
uuid: "cb106c4f38054c378ba28afa9d3b6054"
format_stamp: "Formatted at 2026-08-11 09:08:47 on dist-test-slave-xt4k"
server_key: "9a4d66f898c271031744050f0657e0dc"
server_key_iv: "89e63e2a6c37fdc27dd400d6d19e0766"
server_key_version: "encryptionkey@0"
I20260811 09:08:47.111284 12550 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:08:47.137748 12550 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:08:47.142469 12550 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:08:47.142771 12550 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:08:47.143899 12550 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:08:47.145391 12550 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:08:47.145473 12550 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:08:47.145591 12550 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:08:47.145654 12550 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:08:47.178375 12550 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.129:41847
I20260811 09:08:47.178421 12777 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.129:41847 every 8 connection(s)
I20260811 09:08:47.179713 12550 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/ts-0/data/info.pb
I20260811 09:08:47.185859 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 12550
I20260811 09:08:47.186113 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/ts-0/wal/instance
I20260811 09:08:47.186734 29290 external_mini_cluster.cc:1428] Setting key b0674cd2b2e85b293d6e2f252c7dcaf6
Aug 11 09:08:47 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439327, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.129@KRBTEST.COM for kudu/127.28.154.190@KRBTEST.COM
I20260811 09:08:47.201881 12780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:47.182390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.129:39109 (local address 127.28.154.190:39729)
0811 09:08:47.182940 (+ 550us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:47.182947 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:47.183914 (+ 967us) server_negotiation.cc:407] Connection header received
0811 09:08:47.186072 (+ 2158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:47.186079 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:47.186171 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:47.186293 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:47.188212 (+ 1919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:47.189240 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:47.190866 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:47.191436 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:47.191577 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:47.195285 (+ 3708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:47.195292 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:47.195296 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:08:47.195337 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:47.198422 (+ 3085us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:47.199249 (+ 827us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:47.199255 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:47.199258 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:47.199410 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:47.199989 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:47.199994 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:47.199997 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:47.200284 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:47.200549 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:47.201402 (+ 853us) server_negotiation.cc:299] Negotiation successful
0811 09:08:47.201600 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":416,"thread_start_us":269,"threads_started":1}
I20260811 09:08:47.203279 12786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:47.182946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:39729 (local address 127.28.154.129:39109)
0811 09:08:47.183735 (+ 789us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:47.183782 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:47.185683 (+ 1901us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:47.186591 (+ 908us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:47.186601 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:47.187213 (+ 612us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:08:47.188019 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:47.188049 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:47.189413 (+ 1364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:47.189417 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:47.190702 (+ 1285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:47.190717 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:47.191680 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:47.191685 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:47.191896 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:47.192639 (+ 743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:08:47.192664 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:08:47.195023 (+ 2359us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:08:47.198598 (+ 3575us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:08:47.198607 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:08:47.198623 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:08:47.199119 (+ 496us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:08:47.199553 (+ 434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:08:47.199559 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:08:47.199562 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:08:47.199845 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:08:47.200453 (+ 608us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:08:47.200461 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:08:47.200941 (+ 480us) client_negotiation.cc:769] Sending connection context
0811 09:08:47.201276 (+ 335us) client_negotiation.cc:240] Negotiation successful
0811 09:08:47.201849 (+ 573us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":532,"thread_start_us":274,"threads_started":1}
I20260811 09:08:47.205065 12778 heartbeater.cc:347] Connected to a master server at 127.28.154.190:39729
I20260811 09:08:47.205395 12778 heartbeater.cc:464] Registering TS with master...
I20260811 09:08:47.206169 12778 heartbeater.cc:511] Master 127.28.154.190:39729 requested a full tablet report, sending...
I20260811 09:08:47.208509 12443 ts_manager.cc:194] Registered new tserver with Master: cb106c4f38054c378ba28afa9d3b6054 (127.28.154.129:41847)
I20260811 09:08:47.210651 12443 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.154.129@KRBTEST.COM'} at 127.28.154.129:39109
WARNING: no policy specified for kudu/127.28.154.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.154.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.154.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.130.keytab.
Entry for principal kudu/127.28.154.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.130.keytab.
I20260811 09:08:47.250080 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.154.130:0
--local_ip_for_outbound_sockets=127.28.154.130
--webserver_interface=127.28.154.130
--webserver_port=0
--tserver_master_addrs=127.28.154.190:39729
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.130.keytab
--principal=kudu/127.28.154.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:45223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:08:47.592722 12791 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:08:47.593043 12791 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:08:47.593145 12791 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:08:47.600958 12791 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:08:47.601087 12791 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:08:47.601213 12791 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.154.130
I20260811 09:08:47.611282 12791 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:45223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.130.keytab
--principal=kudu/127.28.154.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.154.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.154.130
--webserver_port=0
--tserver_master_addrs=127.28.154.190:39729
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.154.130
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:08:47.612672 12791 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:08:47.613963 12791 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:08:47.627789 12797 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:08:47.627758 12798 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:08:47.627874 12800 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:08:47.628922 12791 server_base.cc:1034] running on GCE node
I20260811 09:08:47.629572 12791 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:08:47.630522 12791 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:08:47.631755 12791 hybrid_clock.cc:648] HybridClock initialized: now 1786439327631696 us; error 40 us; skew 500 ppm
I20260811 09:08:47.632166 12791 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:08:47 dist-test-slave-xt4k krb5kdc[12376](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439327, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:08:47.635620 12791 init.cc:377] Logged in from keytab as kudu/127.28.154.130@KRBTEST.COM (short username kudu)
I20260811 09:08:47.637266 12791 webserver.cc:460] Webserver started at http://127.28.154.130:35351/ using document root <none> and password file <none>
I20260811 09:08:47.638018 12791 fs_manager.cc:359] Metadata directory not provided
I20260811 09:08:47.638144 12791 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:08:47.638468 12791 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:08:47.641538 12791 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/ts-1/data/instance:
uuid: "f18a79b21ac34f7c8125551a8d71014c"
format_stamp: "Formatted at 2026-08-11 09:08:47 on dist-test-slave-xt4k"
server_key: "3261380062de724519556bc93972b14f"
server_key_iv: "d6bbac2ee37789cf9266d411f737daea"
server_key_version: "encryptionkey@0"
I20260811 09:08:47.642346 12791 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/ts-1/wal/instance:
uuid: "f18a79b21ac34f7c8125551a8d71014c"
format_stamp: "Formatted at 2026-08-11 09:08:47 on dist-test-slave-xt4k"
server_key: "3261380062de724519556bc93972b14f"
server_key_iv: "d6bbac2ee37789cf9266d411f737daea"
server_key_version: "encryptionkey@0"
I20260811 09:08:47.647950 12791 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260811 09:08:47.651660 12810 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:08:47.653158 12791 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 09:08:47.653332 12791 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/ts-1/wal
uuid: "f18a79b21ac34f7c8125551a8d71014c"
format_stamp: "Formatted at 2026-08-11 09:08:47 on dist-test-slave-xt4k"
server_key: "3261380062de724519556bc93972b14f"
server_key_iv: "d6bbac2ee37789cf9266d411f737daea"
server_key_version: "encryptionkey@0"
I20260811 09:08:47.653535 12791 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:08:47.691584 12791 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:08:47.696230 12791 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:08:47.696568 12791 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:08:47.697600 12791 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:08:47.698994 12791 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:08:47.699083 12791 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:08:47.699203 12791 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:08:47.699255 12791 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:08:47.730859 12791 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.130:36935
I20260811 09:08:47.730916 13001 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.130:36935 every 8 connection(s)
I20260811 09:08:47.732425 12791 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/ts-1/data/info.pb
I20260811 09:08:47.738523 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 12791
I20260811 09:08:47.738719 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/ts-1/wal/instance
I20260811 09:08:47.739178 29290 external_mini_cluster.cc:1428] Setting key 184b122a48f4586f337f41e313589b65
Aug 11 09:08:47 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439327, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.130@KRBTEST.COM for kudu/127.28.154.190@KRBTEST.COM
I20260811 09:08:47.753491 13007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:47.735287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.130:51457 (local address 127.28.154.190:39729)
0811 09:08:47.735880 (+ 593us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:47.735887 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:47.736968 (+ 1081us) server_negotiation.cc:407] Connection header received
0811 09:08:47.738907 (+ 1939us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:47.738913 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:47.739008 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:47.739118 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:47.740876 (+ 1758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:47.741747 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:47.743322 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:47.743751 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:47.743866 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:47.747563 (+ 3697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:47.747569 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:47.747572 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:08:47.747608 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:47.750208 (+ 2600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:47.750949 (+ 741us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:47.750953 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:47.750956 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:47.751110 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:47.751549 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:47.751553 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:47.751555 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:47.751830 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:47.751969 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:47.752850 (+ 881us) server_negotiation.cc:299] Negotiation successful
0811 09:08:47.753107 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":364,"thread_start_us":239,"threads_started":1}
I20260811 09:08:47.754565 13012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:47.735980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:39729 (local address 127.28.154.130:51457)
0811 09:08:47.736805 (+ 825us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:47.736847 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:47.738592 (+ 1745us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:47.739376 (+ 784us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:47.739385 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:47.740048 (+ 663us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:08:47.740698 (+ 650us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:47.740721 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:47.741903 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:47.741907 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:47.743172 (+ 1265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:47.743185 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:47.743978 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:47.743985 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:47.744237 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:47.744957 (+ 720us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:08:47.744983 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:08:47.747297 (+ 2314us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:08:47.750345 (+ 3048us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:08:47.750356 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:08:47.750377 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:08:47.750816 (+ 439us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:08:47.751220 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:08:47.751225 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:08:47.751229 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:08:47.751426 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:08:47.751958 (+ 532us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:08:47.751966 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:08:47.752324 (+ 358us) client_negotiation.cc:769] Sending connection context
0811 09:08:47.752648 (+ 324us) client_negotiation.cc:240] Negotiation successful
0811 09:08:47.753308 (+ 660us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":582,"thread_start_us":351,"threads_started":1}
I20260811 09:08:47.756096 13003 heartbeater.cc:347] Connected to a master server at 127.28.154.190:39729
I20260811 09:08:47.756423 13003 heartbeater.cc:464] Registering TS with master...
I20260811 09:08:47.757203 13003 heartbeater.cc:511] Master 127.28.154.190:39729 requested a full tablet report, sending...
I20260811 09:08:47.758785 12443 ts_manager.cc:194] Registered new tserver with Master: f18a79b21ac34f7c8125551a8d71014c (127.28.154.130:36935)
I20260811 09:08:47.759970 12443 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.154.130@KRBTEST.COM'} at 127.28.154.130:51457
WARNING: no policy specified for kudu/127.28.154.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.154.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.154.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.131.keytab.
Entry for principal kudu/127.28.154.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.131.keytab.
I20260811 09:08:47.797775 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.154.131:0
--local_ip_for_outbound_sockets=127.28.154.131
--webserver_interface=127.28.154.131
--webserver_port=0
--tserver_master_addrs=127.28.154.190:39729
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.131.keytab
--principal=kudu/127.28.154.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:45223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:08:48.148039 13021 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:08:48.148329 13021 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:08:48.148483 13021 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:08:48.155812 13021 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:08:48.155939 13021 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:08:48.156057 13021 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.154.131
I20260811 09:08:48.166402 13021 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:45223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.131.keytab
--principal=kudu/127.28.154.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.154.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.154.131
--webserver_port=0
--tserver_master_addrs=127.28.154.190:39729
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.154.131
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:08:48.167716 13021 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:08:48.169087 13021 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:08:48.184257 13033 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:08:48.184286 13030 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:08:48.185295 13038 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:08:48.185699 13021 server_base.cc:1034] running on GCE node
I20260811 09:08:48.186336 13021 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:08:48.187322 13021 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:08:48.188551 13021 hybrid_clock.cc:648] HybridClock initialized: now 1786439328188472 us; error 42 us; skew 500 ppm
I20260811 09:08:48.188887 13021 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:08:48 dist-test-slave-xt4k krb5kdc[12376](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439328, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:08:48.192086 13021 init.cc:377] Logged in from keytab as kudu/127.28.154.131@KRBTEST.COM (short username kudu)
I20260811 09:08:48.193674 13021 webserver.cc:460] Webserver started at http://127.28.154.131:35919/ using document root <none> and password file <none>
I20260811 09:08:48.194424 13021 fs_manager.cc:359] Metadata directory not provided
I20260811 09:08:48.194550 13021 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:08:48.194851 13021 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:08:48.197858 13021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/ts-2/data/instance:
uuid: "7b72dbec666a4e048717860d004c76b3"
format_stamp: "Formatted at 2026-08-11 09:08:48 on dist-test-slave-xt4k"
server_key: "f42d06b0b76673323cddac4d0df209cc"
server_key_iv: "eb38e7207ab67997078066fc0d4a1972"
server_key_version: "encryptionkey@0"
I20260811 09:08:48.198635 13021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/ts-2/wal/instance:
uuid: "7b72dbec666a4e048717860d004c76b3"
format_stamp: "Formatted at 2026-08-11 09:08:48 on dist-test-slave-xt4k"
server_key: "f42d06b0b76673323cddac4d0df209cc"
server_key_iv: "eb38e7207ab67997078066fc0d4a1972"
server_key_version: "encryptionkey@0"
I20260811 09:08:48.203930 13021 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 09:08:48.207500 13048 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:08:48.208859 13021 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 09:08:48.209020 13021 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/ts-2/wal
uuid: "7b72dbec666a4e048717860d004c76b3"
format_stamp: "Formatted at 2026-08-11 09:08:48 on dist-test-slave-xt4k"
server_key: "f42d06b0b76673323cddac4d0df209cc"
server_key_iv: "eb38e7207ab67997078066fc0d4a1972"
server_key_version: "encryptionkey@0"
I20260811 09:08:48.209208 13021 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:08:48.214352 12778 heartbeater.cc:503] Master 127.28.154.190:39729 was elected leader, sending a full tablet report...
I20260811 09:08:48.230075 13021 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:08:48.234828 13021 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:08:48.235143 13021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:08:48.236274 13021 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:08:48.237771 13021 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:08:48.237849 13021 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:08:48.237951 13021 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:08:48.238008 13021 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:08:48.270925 13021 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.131:32793
I20260811 09:08:48.270965 13245 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.131:32793 every 8 connection(s)
I20260811 09:08:48.272313 13021 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/ts-2/data/info.pb
I20260811 09:08:48.275804 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 13021
I20260811 09:08:48.276042 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/ts-2/wal/instance
I20260811 09:08:48.276847 29290 external_mini_cluster.cc:1428] Setting key de072c9a9d4c591816f7866727d823e6
Aug 11 09:08:48 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439328, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.131@KRBTEST.COM for kudu/127.28.154.190@KRBTEST.COM
I20260811 09:08:48.296504 13249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:48.275197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.131:53849 (local address 127.28.154.190:39729)
0811 09:08:48.275751 (+ 554us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:48.275755 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:48.276828 (+ 1073us) server_negotiation.cc:407] Connection header received
0811 09:08:48.280157 (+ 3329us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:48.280163 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:48.280269 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:48.280415 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:48.282276 (+ 1861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:48.283307 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:48.285206 (+ 1899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:48.285654 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:48.285782 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:48.290129 (+ 4347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:48.290135 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:48.290138 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:08:48.290177 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:48.293113 (+ 2936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:48.293983 (+ 870us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:48.293988 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:48.293990 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:48.294117 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:48.294586 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:48.294590 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:48.294592 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:48.294883 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:48.295032 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:48.295839 (+ 807us) server_negotiation.cc:299] Negotiation successful
0811 09:08:48.296084 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":439,"thread_start_us":307,"threads_started":1}
I20260811 09:08:48.297557 13254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:48.275664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:39729 (local address 127.28.154.131:53849)
0811 09:08:48.276661 (+ 997us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:48.276701 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:48.279540 (+ 2839us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:48.280736 (+ 1196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:48.280747 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:48.281440 (+ 693us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:08:48.282091 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:48.282111 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:48.283631 (+ 1520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:48.283636 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:48.285033 (+ 1397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:48.285048 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:48.285904 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:48.285910 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:48.286138 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:48.286879 (+ 741us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:08:48.286904 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:08:48.289841 (+ 2937us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:08:48.293273 (+ 3432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:08:48.293284 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:08:48.293305 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:08:48.293849 (+ 544us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:08:48.294228 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:08:48.294234 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:08:48.294238 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:08:48.294458 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:08:48.295024 (+ 566us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:08:48.295033 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:08:48.295432 (+ 399us) client_negotiation.cc:769] Sending connection context
0811 09:08:48.295701 (+ 269us) client_negotiation.cc:240] Negotiation successful
0811 09:08:48.296306 (+ 605us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":733,"thread_start_us":309,"threads_started":1}
I20260811 09:08:48.299744 13246 heartbeater.cc:347] Connected to a master server at 127.28.154.190:39729
I20260811 09:08:48.300076 13246 heartbeater.cc:464] Registering TS with master...
I20260811 09:08:48.301046 13246 heartbeater.cc:511] Master 127.28.154.190:39729 requested a full tablet report, sending...
I20260811 09:08:48.302815 12443 ts_manager.cc:194] Registered new tserver with Master: 7b72dbec666a4e048717860d004c76b3 (127.28.154.131:32793)
I20260811 09:08:48.303720 12443 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.154.131@KRBTEST.COM'} at 127.28.154.131:53849
I20260811 09:08:48.307315 29290 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.28.154.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.28.154.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.28.154.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/krb5kdc/oryx_127.28.154.129.keytab.
Entry for principal oryx/127.28.154.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/krb5kdc/oryx_127.28.154.129.keytab.
WARNING: no policy specified for oryx/127.28.154.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.28.154.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.28.154.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/krb5kdc/oryx_127.28.154.130.keytab.
Entry for principal oryx/127.28.154.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/krb5kdc/oryx_127.28.154.130.keytab.
WARNING: no policy specified for oryx/127.28.154.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.28.154.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.28.154.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/krb5kdc/oryx_127.28.154.131.keytab.
Entry for principal oryx/127.28.154.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/krb5kdc/oryx_127.28.154.131.keytab.
I20260811 09:08:48.480792 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 12550
Aug 11 09:08:48 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439325, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.154.129@KRBTEST.COM
I20260811 09:08:48.497888 13271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:48.482184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34830 (local address 127.28.154.129:41847)
0811 09:08:48.482837 (+ 653us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:48.482845 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:48.482873 (+ 28us) server_negotiation.cc:407] Connection header received
0811 09:08:48.482936 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:48.482967 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:48.483188 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:48.483350 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:48.484870 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:48.485978 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:48.487285 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:48.487870 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:48.487992 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:48.492127 (+ 4135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:48.492141 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:48.492158 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:08:48.492201 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:48.495011 (+ 2810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:48.495697 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:48.495704 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:48.495707 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:48.495845 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:48.496207 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:48.496211 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:48.496213 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:48.496642 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:48.496968 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:48.497261 (+ 293us) server_negotiation.cc:299] Negotiation successful
0811 09:08:48.497536 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":497,"thread_start_us":330,"threads_started":1}
I20260811 09:08:48.499233 12771 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:08:48.589991 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 12550
I20260811 09:08:48.613273 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 12791
Aug 11 09:08:48 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439325, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.154.130@KRBTEST.COM
I20260811 09:08:48.633416 13274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:48.614722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42290 (local address 127.28.154.130:36935)
0811 09:08:48.615311 (+ 589us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:48.615318 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:48.615346 (+ 28us) server_negotiation.cc:407] Connection header received
0811 09:08:48.615428 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:48.615434 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:48.615651 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:48.615805 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:48.617146 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:48.618400 (+ 1254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:48.619897 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:48.620944 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:48.621066 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:48.625543 (+ 4477us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:48.625556 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:48.625599 (+ 43us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:08:48.625643 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:48.629467 (+ 3824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:48.630368 (+ 901us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:48.630381 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:48.630383 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:48.630623 (+ 240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:48.631168 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:48.631173 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:48.631176 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:48.631649 (+ 473us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:48.631938 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:48.632326 (+ 388us) server_negotiation.cc:299] Negotiation successful
0811 09:08:48.632669 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":448,"thread_start_us":313,"threads_started":1}
I20260811 09:08:48.634950 12995 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:08:48.719403 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 12791
I20260811 09:08:48.742846 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 13021
Aug 11 09:08:48 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439325, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.154.131@KRBTEST.COM
I20260811 09:08:48.762257 13279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:48.743984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46380 (local address 127.28.154.131:32793)
0811 09:08:48.744793 (+ 809us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:48.744803 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:48.744856 (+ 53us) server_negotiation.cc:407] Connection header received
0811 09:08:48.744947 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:48.744955 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:48.745425 (+ 470us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:48.745589 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:48.747072 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:48.748140 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:48.749589 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:48.750366 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:48.750512 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:48.755383 (+ 4871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:48.755407 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:48.755523 (+ 116us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:08:48.755630 (+ 107us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:48.758921 (+ 3291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:48.759687 (+ 766us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:48.759695 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:48.759698 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:48.759865 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:48.760310 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:48.760314 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:48.760316 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:48.760857 (+ 541us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:48.761121 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:48.761473 (+ 352us) server_negotiation.cc:299] Negotiation successful
0811 09:08:48.761858 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":541,"thread_start_us":241,"threads_started":1}
I20260811 09:08:48.763797 13239 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:08:48.854847 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 13021
I20260811 09:08:48.878820 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 12397
I20260811 09:08:48.880210 12493 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:08:48.987246 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 12397
I20260811 09:08:49.014194 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.154.190:39729
--webserver_interface=127.28.154.190
--webserver_port=39147
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab
--principal=kudu/127.28.154.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:45223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.154.190:39729
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:08:49.384021 13283 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:08:49.384371 13283 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:08:49.384495 13283 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:08:49.392272 13283 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:08:49.392439 13283 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:08:49.392791 13283 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:08:49.392889 13283 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:08:49.392963 13283 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:08:49.405237 13283 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:45223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.154.190:39729
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab
--principal=kudu/127.28.154.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.154.190:39729
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.154.190
--webserver_port=39147
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:08:49.406739 13283 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:08:49.408183 13283 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:08:49.420825 13294 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:08:49.420917 13298 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:08:49.420931 13293 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:08:49.421720 13283 server_base.cc:1034] running on GCE node
I20260811 09:08:49.422588 13283 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:08:49.424299 13283 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:08:49.425639 13283 hybrid_clock.cc:648] HybridClock initialized: now 1786439329425436 us; error 193 us; skew 500 ppm
I20260811 09:08:49.426157 13283 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:08:49 dist-test-slave-xt4k krb5kdc[12376](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439329, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:08:49.429533 13283 init.cc:377] Logged in from keytab as kudu/127.28.154.190@KRBTEST.COM (short username kudu)
I20260811 09:08:49.431147 13283 webserver.cc:460] Webserver started at http://127.28.154.190:39147/ using document root <none> and password file <none>
I20260811 09:08:49.431968 13283 fs_manager.cc:359] Metadata directory not provided
I20260811 09:08:49.432150 13283 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:08:49.438131 13283 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 09:08:49.441485 13310 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:08:49.443436 13283 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260811 09:08:49.443641 13283 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/master-0/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/master-0/wal
uuid: "286c0ebf7b5d4e8f9bfd87bf58846b74"
format_stamp: "Formatted at 2026-08-11 09:08:46 on dist-test-slave-xt4k"
server_key: "4b4fdf45fee7bd41e24a8a20bfdf569a"
server_key_iv: "90a041e128e317b0e2efd7a29f655baa"
server_key_version: "encryptionkey@0"
I20260811 09:08:49.444734 13283 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:08:49.459168 13283 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:08:49.464413 13283 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:08:49.464705 13283 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:08:49.483410 13283 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.190:39729
I20260811 09:08:49.483457 13410 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.190:39729 every 8 connection(s)
I20260811 09:08:49.484977 13283 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
I20260811 09:08:49.491979 13411 sys_catalog.cc:263] Verifying existing consensus state
I20260811 09:08:49.493986 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 13283
I20260811 09:08:49.494777 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.154.129:41847
--local_ip_for_outbound_sockets=127.28.154.129
--tserver_master_addrs=127.28.154.190:39729
--webserver_port=39655
--webserver_interface=127.28.154.129
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.129.keytab
--principal=kudu/127.28.154.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:45223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.28.154.129
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/krb5kdc/oryx_127.28.154.129.keytab with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 09:08:49.495729 13411 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 286c0ebf7b5d4e8f9bfd87bf58846b74: Bootstrap starting.
I20260811 09:08:49.516438 13411 log.cc:824] T 00000000000000000000000000000000 P 286c0ebf7b5d4e8f9bfd87bf58846b74: Log is configured to *not* fsync() on all Append() calls
I20260811 09:08:49.525848 13411 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 286c0ebf7b5d4e8f9bfd87bf58846b74: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 09:08:49.526362 13411 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 286c0ebf7b5d4e8f9bfd87bf58846b74: Bootstrap complete.
I20260811 09:08:49.535638 13411 raft_consensus.cc:348] T 00000000000000000000000000000000 P 286c0ebf7b5d4e8f9bfd87bf58846b74 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "286c0ebf7b5d4e8f9bfd87bf58846b74" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 39729 } }
I20260811 09:08:49.536727 13411 raft_consensus.cc:729] T 00000000000000000000000000000000 P 286c0ebf7b5d4e8f9bfd87bf58846b74 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 286c0ebf7b5d4e8f9bfd87bf58846b74, State: Initialized, Role: FOLLOWER
I20260811 09:08:49.537431 13411 consensus_queue.cc:260] T 00000000000000000000000000000000 P 286c0ebf7b5d4e8f9bfd87bf58846b74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "286c0ebf7b5d4e8f9bfd87bf58846b74" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 39729 } }
I20260811 09:08:49.537679 13411 raft_consensus.cc:388] T 00000000000000000000000000000000 P 286c0ebf7b5d4e8f9bfd87bf58846b74 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:08:49.537801 13411 raft_consensus.cc:482] T 00000000000000000000000000000000 P 286c0ebf7b5d4e8f9bfd87bf58846b74 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:08:49.537972 13411 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 286c0ebf7b5d4e8f9bfd87bf58846b74 [term 1 FOLLOWER]: Advancing to term 2
I20260811 09:08:49.542148 13411 raft_consensus.cc:504] T 00000000000000000000000000000000 P 286c0ebf7b5d4e8f9bfd87bf58846b74 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "286c0ebf7b5d4e8f9bfd87bf58846b74" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 39729 } }
I20260811 09:08:49.542781 13411 leader_election.cc:302] T 00000000000000000000000000000000 P 286c0ebf7b5d4e8f9bfd87bf58846b74 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 286c0ebf7b5d4e8f9bfd87bf58846b74; no voters:
I20260811 09:08:49.543418 13411 leader_election.cc:288] T 00000000000000000000000000000000 P 286c0ebf7b5d4e8f9bfd87bf58846b74 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 09:08:49.543622 13418 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 286c0ebf7b5d4e8f9bfd87bf58846b74 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 09:08:49.544646 13418 raft_consensus.cc:686] T 00000000000000000000000000000000 P 286c0ebf7b5d4e8f9bfd87bf58846b74 [term 2 LEADER]: Becoming Leader. State: Replica: 286c0ebf7b5d4e8f9bfd87bf58846b74, State: Running, Role: LEADER
I20260811 09:08:49.545449 13411 sys_catalog.cc:564] T 00000000000000000000000000000000 P 286c0ebf7b5d4e8f9bfd87bf58846b74 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:08:49.545312 13418 consensus_queue.cc:237] T 00000000000000000000000000000000 P 286c0ebf7b5d4e8f9bfd87bf58846b74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "286c0ebf7b5d4e8f9bfd87bf58846b74" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 39729 } }
I20260811 09:08:49.548518 13420 sys_catalog.cc:455] T 00000000000000000000000000000000 P 286c0ebf7b5d4e8f9bfd87bf58846b74 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 286c0ebf7b5d4e8f9bfd87bf58846b74. Latest consensus state: current_term: 2 leader_uuid: "286c0ebf7b5d4e8f9bfd87bf58846b74" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "286c0ebf7b5d4e8f9bfd87bf58846b74" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 39729 } } }
I20260811 09:08:49.549007 13420 sys_catalog.cc:458] T 00000000000000000000000000000000 P 286c0ebf7b5d4e8f9bfd87bf58846b74 [sys.catalog]: This master's current role is: LEADER
I20260811 09:08:49.549244 13419 sys_catalog.cc:455] T 00000000000000000000000000000000 P 286c0ebf7b5d4e8f9bfd87bf58846b74 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "286c0ebf7b5d4e8f9bfd87bf58846b74" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "286c0ebf7b5d4e8f9bfd87bf58846b74" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 39729 } } }
I20260811 09:08:49.549597 13419 sys_catalog.cc:458] T 00000000000000000000000000000000 P 286c0ebf7b5d4e8f9bfd87bf58846b74 [sys.catalog]: This master's current role is: LEADER
I20260811 09:08:49.557946 13431 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:08:49.564675 13431 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:08:49.566452 13431 catalog_manager.cc:1259] Loaded cluster ID: 8a2d35833a964c3f92e20cb489e342bd
I20260811 09:08:49.566664 13431 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:08:49.571048 13431 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:08:49.573148 13431 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 286c0ebf7b5d4e8f9bfd87bf58846b74: Loaded TSK: 0
I20260811 09:08:49.575013 13431 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 09:08:49.874825 13413 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:08:49.875140 13413 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:08:49.875274 13413 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:08:49.883147 13413 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:08:49.883279 13413 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:08:49.883438 13413 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.154.129
I20260811 09:08:49.893539 13413 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:45223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/krb5kdc/oryx_127.28.154.129.keytab
--principal=oryx/127.28.154.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.154.129:41847
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.154.129
--webserver_port=39655
--tserver_master_addrs=127.28.154.190:39729
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.154.129
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:08:49.894925 13413 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:08:49.896239 13413 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:08:49.911101 13463 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:08:49.911502 13461 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:08:49.911972 13465 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:08:49.912128 13413 server_base.cc:1034] running on GCE node
I20260811 09:08:49.912817 13413 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:08:49.913798 13413 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:08:49.915016 13413 hybrid_clock.cc:648] HybridClock initialized: now 1786439329914957 us; error 51 us; skew 500 ppm
I20260811 09:08:49.915329 13413 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:08:49 dist-test-slave-xt4k krb5kdc[12376](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439329, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.154.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:08:49.918450 13413 init.cc:377] Logged in from keytab as oryx/127.28.154.129@KRBTEST.COM (short username oryx)
I20260811 09:08:49.920017 13413 webserver.cc:460] Webserver started at http://127.28.154.129:39655/ using document root <none> and password file <none>
I20260811 09:08:49.920753 13413 fs_manager.cc:359] Metadata directory not provided
I20260811 09:08:49.920878 13413 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:08:49.926878 13413 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.002s
I20260811 09:08:49.929658 13477 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:08:49.931026 13413 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 09:08:49.931183 13413 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/ts-0/wal
uuid: "cb106c4f38054c378ba28afa9d3b6054"
format_stamp: "Formatted at 2026-08-11 09:08:47 on dist-test-slave-xt4k"
server_key: "9a4d66f898c271031744050f0657e0dc"
server_key_iv: "89e63e2a6c37fdc27dd400d6d19e0766"
server_key_version: "encryptionkey@0"
I20260811 09:08:49.932165 13413 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:08:49.956296 13413 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:08:49.960978 13413 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:08:49.961215 13413 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:08:49.962312 13413 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:08:49.963790 13413 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:08:49.963891 13413 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:08:49.963994 13413 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:08:49.964068 13413 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:08:49.996276 13413 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.129:41847
I20260811 09:08:49.996368 13679 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.129:41847 every 8 connection(s)
I20260811 09:08:49.997828 13413 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/ts-0/data/info.pb
I20260811 09:08:50.003906 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 13413
I20260811 09:08:50.004575 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.154.130:36935
--local_ip_for_outbound_sockets=127.28.154.130
--tserver_master_addrs=127.28.154.190:39729
--webserver_port=35351
--webserver_interface=127.28.154.130
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.130.keytab
--principal=kudu/127.28.154.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:45223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.28.154.130
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/krb5kdc/oryx_127.28.154.130.keytab with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 09:08:50 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.154.129@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:08:50 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.154.129@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:08:50.017899 13685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:50.000950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:39729 (local address 127.28.154.129:55625)
0811 09:08:50.001875 (+ 925us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:50.001914 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:50.003725 (+ 1811us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:50.005305 (+ 1580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:50.005317 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:50.005932 (+ 615us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:08:50.006487 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:50.006507 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:50.008565 (+ 2058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:50.008570 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:50.010061 (+ 1491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:50.010078 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:50.011211 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:50.011220 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:50.011490 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:50.012227 (+ 737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:08:50.012258 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:08:50.016280 (+ 4022us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.154.190:39729: Server oryx/127.28.154.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":543,"thread_start_us":333,"threads_started":1}
W20260811 09:08:50.017926 13684 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:08:50.000915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.129:55625 (local address 127.28.154.190:39729)
0811 09:08:50.001768 (+ 853us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:50.001786 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:50.002046 (+ 260us) server_negotiation.cc:407] Connection header received
0811 09:08:50.004317 (+ 2271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:50.004331 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:50.004753 (+ 422us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:50.005109 (+ 356us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:50.006701 (+ 1592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:50.008327 (+ 1626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:50.010241 (+ 1914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:50.010936 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:50.011164 (+ 228us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:50.016727 (+ 5563us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.154.129:55625: BlockingRecv error: recv got EOF from 127.28.154.129:55625 (error 108)
Metrics: {"server-negotiator.queue_time_us":540,"thread_start_us":293,"threads_started":1}
W20260811 09:08:50.018909 13680 heartbeater.cc:644] Failed to heartbeat to 127.28.154.190:39729 (0 consecutive failures): Not authorized: Failed to ping master at 127.28.154.190:39729: Client connection negotiation failed: client connection to 127.28.154.190:39729: Server oryx/127.28.154.190@KRBTEST.COM not found in Kerberos database
Aug 11 09:08:50 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.154.129@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:08:50 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.154.129@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:08:50.028198 13685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:50.019597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:39729 (local address 127.28.154.129:53525)
0811 09:08:50.019787 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:50.019806 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:50.019924 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:50.020374 (+ 450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:50.020380 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:50.020769 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:08:50.020923 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:50.020933 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:50.022057 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:50.022061 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:50.023162 (+ 1101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:50.023177 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:50.023876 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:50.023882 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:50.024059 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:50.024767 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:08:50.024786 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:08:50.027882 (+ 3096us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.154.190:39729: Server oryx/127.28.154.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":48}
W20260811 09:08:50.028365 13684 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:08:50.019766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.129:53525 (local address 127.28.154.190:39729)
0811 09:08:50.019940 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:50.019946 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:50.020022 (+ 76us) server_negotiation.cc:407] Connection header received
0811 09:08:50.020144 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:50.020150 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:50.020221 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:50.020304 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:50.021087 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:50.021904 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:50.023325 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:50.023729 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:50.023846 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:50.028131 (+ 4285us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.154.129:53525: BlockingRecv error: recv got EOF from 127.28.154.129:53525 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 11 09:08:50 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.154.129@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:08:50 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.154.129@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:08:50.038275 13685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:50.029297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:39729 (local address 127.28.154.129:43007)
0811 09:08:50.029455 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:50.029474 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:50.029571 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:50.029932 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:50.029937 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:50.030249 (+ 312us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:08:50.030403 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:50.030412 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:50.031489 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:50.031493 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:50.032609 (+ 1116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:50.032623 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:50.033454 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:50.033460 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:50.033648 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:50.034282 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:08:50.034298 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:08:50.037955 (+ 3657us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.154.190:39729: Server oryx/127.28.154.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":48}
W20260811 09:08:50.038453 13684 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:08:50.029466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.129:43007 (local address 127.28.154.190:39729)
0811 09:08:50.029662 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:50.029668 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:50.029685 (+ 17us) server_negotiation.cc:407] Connection header received
0811 09:08:50.029732 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:50.029735 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:50.029804 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:50.029886 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:50.030578 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:50.031337 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:50.032762 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:50.033287 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:50.033424 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:50.038212 (+ 4788us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.154.129:43007: BlockingRecv error: recv got EOF from 127.28.154.129:43007 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
W20260811 09:08:50.038825 13680 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 09:08:50.347616 13686 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:08:50.347908 13686 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:08:50.348014 13686 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:08:50.355527 13686 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:08:50.355656 13686 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:08:50.355773 13686 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.154.130
I20260811 09:08:50.366453 13686 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:45223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/krb5kdc/oryx_127.28.154.130.keytab
--principal=oryx/127.28.154.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.154.130:36935
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.154.130
--webserver_port=35351
--tserver_master_addrs=127.28.154.190:39729
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.154.130
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:08:50.367847 13686 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:08:50.369227 13686 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:08:50.382973 13699 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:08:50.383031 13696 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:08:50.383107 13694 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:08:50.383765 13686 server_base.cc:1034] running on GCE node
I20260811 09:08:50.384506 13686 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:08:50.385528 13686 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:08:50.386749 13686 hybrid_clock.cc:648] HybridClock initialized: now 1786439330386702 us; error 33 us; skew 500 ppm
I20260811 09:08:50.387104 13686 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:08:50 dist-test-slave-xt4k krb5kdc[12376](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439330, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.154.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:08:50.390403 13686 init.cc:377] Logged in from keytab as oryx/127.28.154.130@KRBTEST.COM (short username oryx)
I20260811 09:08:50.392319 13686 webserver.cc:460] Webserver started at http://127.28.154.130:35351/ using document root <none> and password file <none>
I20260811 09:08:50.393258 13686 fs_manager.cc:359] Metadata directory not provided
I20260811 09:08:50.393405 13686 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:08:50.399616 13686 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.005s
I20260811 09:08:50.402612 13708 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:08:50.404217 13686 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 09:08:50.404430 13686 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/ts-1/wal
uuid: "f18a79b21ac34f7c8125551a8d71014c"
format_stamp: "Formatted at 2026-08-11 09:08:47 on dist-test-slave-xt4k"
server_key: "3261380062de724519556bc93972b14f"
server_key_iv: "d6bbac2ee37789cf9266d411f737daea"
server_key_version: "encryptionkey@0"
I20260811 09:08:50.405457 13686 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:08:50.422928 13686 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:08:50.427371 13686 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:08:50.427618 13686 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:08:50.428618 13686 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:08:50.429970 13686 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:08:50.430060 13686 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:08:50.430141 13686 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:08:50.430198 13686 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:08:50.461506 13686 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.130:36935
I20260811 09:08:50.461580 13905 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.130:36935 every 8 connection(s)
I20260811 09:08:50.462951 13686 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/ts-1/data/info.pb
I20260811 09:08:50.463297 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 13686
I20260811 09:08:50.464073 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.154.131:32793
--local_ip_for_outbound_sockets=127.28.154.131
--tserver_master_addrs=127.28.154.190:39729
--webserver_port=35919
--webserver_interface=127.28.154.131
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.131.keytab
--principal=kudu/127.28.154.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:45223
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.28.154.131
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/krb5kdc/oryx_127.28.154.131.keytab with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 09:08:50 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.154.130@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:08:50 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.154.130@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM, Server not found in Kerberos database
W20260811 09:08:50.479913 13684 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:08:50.466068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.130:54547 (local address 127.28.154.190:39729)
0811 09:08:50.466253 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:50.466257 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:50.467575 (+ 1318us) server_negotiation.cc:407] Connection header received
0811 09:08:50.469594 (+ 2019us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:50.469600 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:50.469684 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:50.469772 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:50.471399 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:50.472282 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:50.474002 (+ 1720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:50.474420 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:50.474588 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:50.479655 (+ 5067us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.154.130:54547: BlockingRecv error: recv got EOF from 127.28.154.130:54547 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 09:08:50.480727 13909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:50.466433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:39729 (local address 127.28.154.130:54547)
0811 09:08:50.467289 (+ 856us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:50.467338 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:50.469302 (+ 1964us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:50.470042 (+ 740us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:50.470050 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:50.470608 (+ 558us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:08:50.471221 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:50.471240 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:50.472508 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:50.472512 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:50.473845 (+ 1333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:50.473859 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:50.474684 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:50.474692 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:50.474929 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:50.475654 (+ 725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:08:50.475677 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:08:50.479439 (+ 3762us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.154.190:39729: Server oryx/127.28.154.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":568,"thread_start_us":345,"threads_started":1}
W20260811 09:08:50.481529 13906 heartbeater.cc:644] Failed to heartbeat to 127.28.154.190:39729 (0 consecutive failures): Not authorized: Failed to ping master at 127.28.154.190:39729: Client connection negotiation failed: client connection to 127.28.154.190:39729: Server oryx/127.28.154.190@KRBTEST.COM not found in Kerberos database
Aug 11 09:08:50 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.154.130@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:08:50 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.154.130@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:08:50.492995 13909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:50.482315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:39729 (local address 127.28.154.130:40009)
0811 09:08:50.482522 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:50.482571 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:50.482799 (+ 228us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:50.483286 (+ 487us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:50.483290 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:50.483827 (+ 537us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:08:50.484021 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:50.484032 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:50.485555 (+ 1523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:50.485560 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:50.486971 (+ 1411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:50.486988 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:50.487945 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:50.487952 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:50.488140 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:50.488942 (+ 802us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:08:50.488970 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:08:50.492702 (+ 3732us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.154.190:39729: Server oryx/127.28.154.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":60}
W20260811 09:08:50.493315 13684 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:08:50.482465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.130:40009 (local address 127.28.154.190:39729)
0811 09:08:50.482709 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:50.482715 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:50.482737 (+ 22us) server_negotiation.cc:407] Connection header received
0811 09:08:50.483042 (+ 305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:50.483047 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:50.483127 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:50.483230 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:50.484177 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:50.485394 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:50.487125 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:50.487780 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:50.487907 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:50.493037 (+ 5130us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.154.130:40009: BlockingRecv error: recv got EOF from 127.28.154.130:40009 (error 108)
Metrics: {"server-negotiator.queue_time_us":124}
Aug 11 09:08:50 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.154.130@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:08:50 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.154.130@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:08:50.502331 13909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:50.494051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:39729 (local address 127.28.154.130:38091)
0811 09:08:50.494244 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:50.494263 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:50.494377 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:50.494825 (+ 448us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:50.494829 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:50.495117 (+ 288us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:08:50.495270 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:50.495279 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:50.496704 (+ 1425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:50.496708 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:50.497624 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:50.497636 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:50.498632 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:50.498636 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:50.498884 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:50.499468 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:08:50.499489 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:08:50.502113 (+ 2624us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.154.190:39729: Server oryx/127.28.154.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":62}
W20260811 09:08:50.502699 13906 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 09:08:50.503041 13684 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:08:50.494151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.130:38091 (local address 127.28.154.190:39729)
0811 09:08:50.494302 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:50.494307 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:50.494326 (+ 19us) server_negotiation.cc:407] Connection header received
0811 09:08:50.494523 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:50.494529 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:50.494669 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:50.494791 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:50.495391 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:50.496573 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:50.497825 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:50.498471 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:50.498604 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:50.502397 (+ 3793us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.154.130:38091: BlockingRecv error: recv got EOF from 127.28.154.130:38091 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
W20260811 09:08:50.817658 13912 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:08:50.817952 13912 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:08:50.818073 13912 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:08:50.825574 13912 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:08:50.825697 13912 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:08:50.825824 13912 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.154.131
I20260811 09:08:50.835825 13912 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:45223
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/krb5kdc/oryx_127.28.154.131.keytab
--principal=oryx/127.28.154.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.154.131:32793
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.154.131
--webserver_port=35919
--tserver_master_addrs=127.28.154.190:39729
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.154.131
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:08:50.837328 13912 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:08:50.838786 13912 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:08:50.852805 13919 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:08:50.852770 13920 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:08:50.852849 13923 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:08:50.853642 13912 server_base.cc:1034] running on GCE node
I20260811 09:08:50.854313 13912 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:08:50.855376 13912 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:08:50.856603 13912 hybrid_clock.cc:648] HybridClock initialized: now 1786439330856546 us; error 50 us; skew 500 ppm
I20260811 09:08:50.856953 13912 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:08:50 dist-test-slave-xt4k krb5kdc[12376](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439330, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.154.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:08:50.860422 13912 init.cc:377] Logged in from keytab as oryx/127.28.154.131@KRBTEST.COM (short username oryx)
I20260811 09:08:50.861963 13912 webserver.cc:460] Webserver started at http://127.28.154.131:35919/ using document root <none> and password file <none>
I20260811 09:08:50.862708 13912 fs_manager.cc:359] Metadata directory not provided
I20260811 09:08:50.862840 13912 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:08:50.868886 13912 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 09:08:50.871981 13935 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:08:50.873596 13912 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 09:08:50.873785 13912 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/ts-2/wal
uuid: "7b72dbec666a4e048717860d004c76b3"
format_stamp: "Formatted at 2026-08-11 09:08:48 on dist-test-slave-xt4k"
server_key: "f42d06b0b76673323cddac4d0df209cc"
server_key_iv: "eb38e7207ab67997078066fc0d4a1972"
server_key_version: "encryptionkey@0"
I20260811 09:08:50.874780 13912 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:08:50.907624 13912 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:08:50.912075 13912 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:08:50.912407 13912 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:08:50.913444 13912 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:08:50.914894 13912 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:08:50.915005 13912 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:08:50.915119 13912 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:08:50.915194 13912 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:08:50.947551 13912 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.131:32793
I20260811 09:08:50.947654 14148 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.131:32793 every 8 connection(s)
I20260811 09:08:50.949229 13912 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786439250192320-29290-0/minicluster-data/ts-2/data/info.pb
I20260811 09:08:50.955274 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 13912
Aug 11 09:08:50 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.154.131@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:08:50 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.154.131@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM, Server not found in Kerberos database
W20260811 09:08:50.966897 13684 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:08:50.951821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.131:35737 (local address 127.28.154.190:39729)
0811 09:08:50.952068 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:50.952075 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:50.953351 (+ 1276us) server_negotiation.cc:407] Connection header received
0811 09:08:50.955238 (+ 1887us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:50.955243 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:50.955331 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:50.955432 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:50.957615 (+ 2183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:50.958396 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:50.960588 (+ 2192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:50.960990 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:50.961098 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:50.966598 (+ 5500us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.154.131:35737: BlockingRecv error: recv got EOF from 127.28.154.131:35737 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
I20260811 09:08:50.967674 14152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:50.952401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:39729 (local address 127.28.154.131:35737)
0811 09:08:50.953173 (+ 772us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:50.953215 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:50.954913 (+ 1698us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:50.955812 (+ 899us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:50.955826 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:50.956550 (+ 724us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:08:50.957420 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:50.957442 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:50.958571 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:50.958575 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:50.960459 (+ 1884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:50.960472 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:50.961222 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:50.961230 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:50.961450 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:50.962174 (+ 724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:08:50.962206 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:08:50.966353 (+ 4147us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.154.190:39729: Server oryx/127.28.154.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":306,"threads_started":1}
W20260811 09:08:50.968437 14150 heartbeater.cc:644] Failed to heartbeat to 127.28.154.190:39729 (0 consecutive failures): Not authorized: Failed to ping master at 127.28.154.190:39729: Client connection negotiation failed: client connection to 127.28.154.190:39729: Server oryx/127.28.154.190@KRBTEST.COM not found in Kerberos database
I20260811 09:08:50.972082 14154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:50.956406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56488 (local address 127.28.154.190:39729)
0811 09:08:50.957341 (+ 935us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:50.957345 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:50.957359 (+ 14us) server_negotiation.cc:407] Connection header received
0811 09:08:50.957402 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:50.957405 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:50.957461 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:50.957559 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:50.959092 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:50.960223 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:50.961517 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:50.962053 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:50.962180 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:50.964798 (+ 2618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:08:50.964810 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:08:50.964835 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:08:50.964888 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:08:50.968499 (+ 3611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:50.969328 (+ 829us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:50.969337 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:50.969346 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:50.969609 (+ 263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:08:50.970052 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:08:50.970056 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:08:50.970058 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:08:50.970670 (+ 612us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:08:50.971026 (+ 356us) server_negotiation.cc:1035] Waiting for connection context
0811 09:08:50.971437 (+ 411us) server_negotiation.cc:299] Negotiation successful
0811 09:08:50.971788 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":845,"thread_start_us":751,"threads_started":1}
Aug 11 09:08:50 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.154.131@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:08:50 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.154.131@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:08:50.978336 14152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:50.969092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:39729 (local address 127.28.154.131:50013)
0811 09:08:50.969270 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:50.969297 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:50.969381 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:50.969698 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:50.969702 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:50.970112 (+ 410us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:08:50.970255 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:50.970264 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:50.971801 (+ 1537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:50.971805 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:50.973069 (+ 1264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:50.973085 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:50.974134 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:50.974138 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:50.974277 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:50.975067 (+ 790us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:08:50.975090 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:08:50.978024 (+ 2934us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.154.190:39729: Server oryx/127.28.154.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":46}
W20260811 09:08:50.978472 13684 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:08:50.969232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.131:50013 (local address 127.28.154.190:39729)
0811 09:08:50.969393 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:50.969403 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:50.969420 (+ 17us) server_negotiation.cc:407] Connection header received
0811 09:08:50.969472 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:50.969476 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:50.969544 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:50.969643 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:50.970388 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:50.971462 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:50.973241 (+ 1779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:50.973959 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:50.974109 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:50.978235 (+ 4126us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.154.131:50013: BlockingRecv error: recv got EOF from 127.28.154.131:50013 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 11 09:08:50 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.154.131@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:08:50 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.154.131@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:08:50.986562 14152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:50.979170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:39729 (local address 127.28.154.131:48251)
0811 09:08:50.979330 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:50.979348 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:50.979425 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:50.979800 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:50.979803 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:50.980149 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:08:50.980301 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:50.980309 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:50.981330 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:50.981335 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:50.982280 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:50.982292 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:50.982963 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:50.982967 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:50.983096 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:50.983663 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:08:50.983679 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:08:50.986344 (+ 2665us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.154.190:39729: Server oryx/127.28.154.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":46}
W20260811 09:08:50.986869 14150 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 09:08:50.987082 13684 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:08:50.979271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.131:48251 (local address 127.28.154.190:39729)
0811 09:08:50.979492 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:50.979496 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:50.979511 (+ 15us) server_negotiation.cc:407] Connection header received
0811 09:08:50.979553 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:50.979556 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:50.979621 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:50.979727 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:50.980457 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:50.981190 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:50.982430 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:50.982837 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:50.982937 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:50.986548 (+ 3611us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.154.131:48251: BlockingRecv error: recv got EOF from 127.28.154.131:48251 (error 108)
Metrics: {"server-negotiator.queue_time_us":121}
Aug 11 09:08:51 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.154.129@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:08:51 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.154.129@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:08:51.048350 14156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:51.039368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:39729 (local address 127.28.154.129:49675)
0811 09:08:51.039849 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:51.039866 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:51.039987 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:51.040311 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:51.040315 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:51.040865 (+ 550us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:08:51.041068 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:51.041079 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:51.042145 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:51.042149 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:51.043344 (+ 1195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:51.043358 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:51.044106 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:51.044111 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:51.044245 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:51.044900 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:08:51.044919 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:08:51.047987 (+ 3068us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.154.190:39729: Server oryx/127.28.154.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":250,"threads_started":1}
W20260811 09:08:51.048520 13684 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:08:51.039474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.129:49675 (local address 127.28.154.190:39729)
0811 09:08:51.039715 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:51.039722 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:51.039938 (+ 216us) server_negotiation.cc:407] Connection header received
0811 09:08:51.040084 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:51.040087 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:51.040151 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:51.040231 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:51.041225 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:51.041999 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:51.043478 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:51.043916 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:51.044011 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:51.048260 (+ 4249us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.154.129:49675: BlockingRecv error: recv got EOF from 127.28.154.129:49675 (error 108)
Metrics: {"server-negotiator.queue_time_us":90}
Aug 11 09:08:51 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.154.130@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:08:51 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.154.130@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:08:51.515075 14158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:51.503425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:39729 (local address 127.28.154.130:33627)
0811 09:08:51.503981 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:51.503997 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:51.504145 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:51.504504 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:51.504509 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:51.505006 (+ 497us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:08:51.505211 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:51.505221 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:51.506259 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:51.506263 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:51.507400 (+ 1137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:51.507415 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:51.508165 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:51.508171 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:51.508325 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:51.511697 (+ 3372us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:08:51.511718 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:08:51.514789 (+ 3071us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.154.190:39729: Server oryx/127.28.154.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":310,"threads_started":1}
W20260811 09:08:51.515358 13684 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:08:51.503493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.130:33627 (local address 127.28.154.190:39729)
0811 09:08:51.503719 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:51.503726 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:51.504095 (+ 369us) server_negotiation.cc:407] Connection header received
0811 09:08:51.504274 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:51.504278 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:51.504370 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:51.504460 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:51.505357 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:51.506109 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:51.507548 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:51.507974 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:51.508088 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:51.515092 (+ 7004us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.154.130:33627: BlockingRecv error: recv got EOF from 127.28.154.130:33627 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 11 09:08:51 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.154.131@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:08:51 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.154.131@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:08:51.996776 14159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:51.987562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:39729 (local address 127.28.154.131:42551)
0811 09:08:51.988158 (+ 596us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:51.988177 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:51.988320 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:51.988768 (+ 448us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:51.988772 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:51.989367 (+ 595us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:08:51.989617 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:51.989629 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:51.990701 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:51.990705 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:51.991682 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:51.991693 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:51.992480 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:51.992485 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:51.992619 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:51.993253 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:08:51.993272 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:08:51.996469 (+ 3197us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.154.190:39729: Server oryx/127.28.154.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":332,"threads_started":1}
W20260811 09:08:51.996959 13684 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:08:51.987680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.131:42551 (local address 127.28.154.190:39729)
0811 09:08:51.987866 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:51.987872 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:51.988255 (+ 383us) server_negotiation.cc:407] Connection header received
0811 09:08:51.988520 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:51.988525 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:51.988611 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:51.988696 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:51.989794 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:51.990556 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:51.991839 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:51.992264 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:51.992402 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:51.996711 (+ 4309us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.154.131:42551: BlockingRecv error: recv got EOF from 127.28.154.131:42551 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 11 09:08:52 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.154.129@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:08:52 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.154.129@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:08:52.057727 14162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:52.049435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:39729 (local address 127.28.154.129:50727)
0811 09:08:52.049993 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:52.050010 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:52.050162 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:52.050486 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:52.050491 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:52.050939 (+ 448us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:08:52.051116 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:52.051126 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:52.052160 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:52.052165 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:52.053102 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:52.053115 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:52.053808 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:52.053815 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:52.053969 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:52.054573 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:08:52.054591 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:08:52.057453 (+ 2862us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.154.190:39729: Server oryx/127.28.154.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":278,"threads_started":1}
W20260811 09:08:52.057916 13684 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:08:52.049584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.129:50727 (local address 127.28.154.190:39729)
0811 09:08:52.049824 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:52.049831 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:52.050105 (+ 274us) server_negotiation.cc:407] Connection header received
0811 09:08:52.050285 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:52.050291 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:52.050359 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:52.050442 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:52.051264 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:52.052025 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:52.053241 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:52.053662 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:52.053770 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:52.057678 (+ 3908us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.154.129:50727: BlockingRecv error: recv got EOF from 127.28.154.129:50727 (error 108)
Metrics: {"server-negotiator.queue_time_us":98}
Aug 11 09:08:52 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.154.130@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:08:52 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.154.130@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:08:52.525174 14164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:52.516190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:39729 (local address 127.28.154.130:55557)
0811 09:08:52.516769 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:52.516787 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:52.516924 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:52.517283 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:52.517287 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:52.517764 (+ 477us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:08:52.517952 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:52.517963 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:52.519123 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:52.519128 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:52.520099 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:52.520114 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:52.521004 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:52.521011 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:52.521195 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:52.521991 (+ 796us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:08:52.522012 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:08:52.524920 (+ 2908us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.154.190:39729: Server oryx/127.28.154.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":349,"threads_started":1}
W20260811 09:08:52.525357 13684 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:08:52.516376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.130:55557 (local address 127.28.154.190:39729)
0811 09:08:52.516554 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:52.516559 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:52.516891 (+ 332us) server_negotiation.cc:407] Connection header received
0811 09:08:52.517038 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:52.517042 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:52.517111 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:52.517205 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:52.518127 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:52.518954 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:52.520247 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:52.520833 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:52.520963 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:52.525136 (+ 4173us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.154.130:55557: BlockingRecv error: recv got EOF from 127.28.154.130:55557 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 11 09:08:53 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.154.131@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:08:53 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.154.131@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:08:53.006309 14168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:52.997863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:39729 (local address 127.28.154.131:57017)
0811 09:08:52.998362 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:52.998380 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:52.998526 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:52.998900 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:52.998904 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:52.999380 (+ 476us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:08:52.999569 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:52.999580 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:53.000586 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:53.000590 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:53.001585 (+ 995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:53.001599 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:53.002292 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:53.002296 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:53.002447 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:53.003114 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:08:53.003134 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:08:53.006056 (+ 2922us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.154.190:39729: Server oryx/127.28.154.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":329,"threads_started":1}
W20260811 09:08:53.006630 13684 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:08:52.998012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.131:57017 (local address 127.28.154.190:39729)
0811 09:08:52.998191 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:52.998198 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:52.998456 (+ 258us) server_negotiation.cc:407] Connection header received
0811 09:08:52.998656 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:52.998661 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:52.998744 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:52.998846 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:52.999704 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:53.000460 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:53.001732 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:53.002169 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:53.002264 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:53.006324 (+ 4060us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.154.131:57017: BlockingRecv error: recv got EOF from 127.28.154.131:57017 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 11 09:08:53 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.154.129@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:08:53 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.154.129@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:08:53.067705 14169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:53.058920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:39729 (local address 127.28.154.129:38673)
0811 09:08:53.059529 (+ 609us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:53.059550 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:53.059711 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:53.060079 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:53.060083 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:53.060591 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:08:53.060811 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:53.060826 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:53.062139 (+ 1313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:53.062143 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:53.063055 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:53.063077 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:53.063738 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:53.063742 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:53.063864 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:53.064517 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:08:53.064536 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:08:53.067352 (+ 2816us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.154.190:39729: Server oryx/127.28.154.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":338,"threads_started":1}
W20260811 09:08:53.067833 13684 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:08:53.059047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.129:38673 (local address 127.28.154.190:39729)
0811 09:08:53.059317 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:53.059324 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:53.059679 (+ 355us) server_negotiation.cc:407] Connection header received
0811 09:08:53.059831 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:53.059837 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:53.059924 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:53.060022 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:53.060998 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:53.061987 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:53.063207 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:53.063621 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:53.063731 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:53.067603 (+ 3872us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.154.129:38673: BlockingRecv error: recv got EOF from 127.28.154.129:38673 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Aug 11 09:08:53 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.154.130@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:08:53 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.154.130@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:08:53.534509 14170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:53.526115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:39729 (local address 127.28.154.130:45469)
0811 09:08:53.526546 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:53.526586 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:53.526705 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:53.527053 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:53.527057 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:53.527461 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:08:53.527648 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:53.527659 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:53.528830 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:53.528834 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:53.529767 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:53.529780 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:53.530442 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:53.530446 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:53.530604 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:53.531190 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:08:53.531207 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:08:53.534235 (+ 3028us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.154.190:39729: Server oryx/127.28.154.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":227,"threads_started":1}
W20260811 09:08:53.534611 13684 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:08:53.526179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.130:45469 (local address 127.28.154.190:39729)
0811 09:08:53.526407 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:53.526414 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:53.526693 (+ 279us) server_negotiation.cc:407] Connection header received
0811 09:08:53.526822 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:53.526826 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:53.526894 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:53.527001 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:53.527808 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:53.528683 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:53.529910 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:53.530330 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:53.530461 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:53.534407 (+ 3946us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.154.130:45469: BlockingRecv error: recv got EOF from 127.28.154.130:45469 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
Aug 11 09:08:54 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.154.131@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:08:54 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.154.131@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:08:54.015936 14175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:54.007284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:39729 (local address 127.28.154.131:39159)
0811 09:08:54.007768 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:54.007785 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:54.007904 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:54.008242 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:54.008245 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:54.008700 (+ 455us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:08:54.008878 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:54.008888 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:54.009951 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:54.009957 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:54.010939 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:54.010955 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:54.011634 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:54.011639 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:54.011765 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:54.012373 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:08:54.012397 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:08:54.015528 (+ 3131us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.154.190:39729: Server oryx/127.28.154.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":266,"threads_started":1}
W20260811 09:08:54.015959 13684 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:08:54.007415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.131:39159 (local address 127.28.154.190:39729)
0811 09:08:54.007583 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:54.007588 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:54.007898 (+ 310us) server_negotiation.cc:407] Connection header received
0811 09:08:54.008007 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:54.008011 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:54.008080 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:54.008179 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:54.009040 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:54.009818 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:54.011080 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:54.011493 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:54.011597 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:54.015748 (+ 4151us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.154.131:39159: BlockingRecv error: recv got EOF from 127.28.154.131:39159 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 11 09:08:54 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.154.129@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:08:54 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.154.129@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:08:54.077266 14178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:54.068772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:39729 (local address 127.28.154.129:49973)
0811 09:08:54.069279 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:54.069295 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:54.069439 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:54.069776 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:54.069780 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:54.070185 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:08:54.070341 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:54.070351 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:54.071376 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:54.071381 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:54.072410 (+ 1029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:54.072423 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:54.073140 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:54.073147 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:54.073314 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:54.073934 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:08:54.073952 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:08:54.076926 (+ 2974us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.154.190:39729: Server oryx/127.28.154.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":276,"threads_started":1}
W20260811 09:08:54.077417 13684 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:08:54.068849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.129:49973 (local address 127.28.154.190:39729)
0811 09:08:54.069113 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:54.069120 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:54.069386 (+ 266us) server_negotiation.cc:407] Connection header received
0811 09:08:54.069564 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:54.069568 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:54.069631 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:54.069717 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:54.070505 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:54.071241 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:54.072567 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:54.072990 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:54.073108 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:54.077166 (+ 4058us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.154.129:49973: BlockingRecv error: recv got EOF from 127.28.154.129:49973 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
Aug 11 09:08:54 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.154.130@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:08:54 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.154.130@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:08:54.543758 14180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:54.535588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:39729 (local address 127.28.154.130:41369)
0811 09:08:54.536083 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:54.536100 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:54.536215 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:54.536565 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:54.536571 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:54.536981 (+ 410us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:08:54.537141 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:54.537150 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:54.538188 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:54.538192 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:54.539186 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:54.539198 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:54.539917 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:54.539922 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:54.540057 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:54.540661 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:08:54.540678 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:08:54.543484 (+ 2806us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.154.190:39729: Server oryx/127.28.154.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":292,"threads_started":1}
W20260811 09:08:54.544028 13684 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:08:54.535624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.130:41369 (local address 127.28.154.190:39729)
0811 09:08:54.535845 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:54.535853 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:54.536175 (+ 322us) server_negotiation.cc:407] Connection header received
0811 09:08:54.536309 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:54.536313 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:54.536409 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:54.536491 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:54.537273 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:54.538060 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:54.539325 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:54.539784 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:54.539879 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:54.543745 (+ 3866us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.154.130:41369: BlockingRecv error: recv got EOF from 127.28.154.130:41369 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 11 09:08:55 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.154.131@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:08:55 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.154.131@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:08:55.025944 14183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:55.017062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:39729 (local address 127.28.154.131:54589)
0811 09:08:55.017556 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:55.017580 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:55.017729 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:55.018068 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:55.018073 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:55.018433 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:08:55.018614 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:55.018624 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:55.019709 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:55.019713 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:55.020837 (+ 1124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:55.020852 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:55.021677 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:55.021683 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:55.021983 (+ 300us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:55.022694 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:08:55.022715 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:08:55.025705 (+ 2990us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.154.190:39729: Server oryx/127.28.154.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":252,"threads_started":1}
W20260811 09:08:55.026249 13684 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:08:55.017204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.131:54589 (local address 127.28.154.190:39729)
0811 09:08:55.017398 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:55.017403 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:55.017656 (+ 253us) server_negotiation.cc:407] Connection header received
0811 09:08:55.017856 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:55.017861 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:55.017943 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:55.018039 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:55.018775 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:55.019571 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:55.020982 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:55.021512 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:55.021651 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:55.025989 (+ 4338us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.154.131:54589: BlockingRecv error: recv got EOF from 127.28.154.131:54589 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 11 09:08:55 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.154.129@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:08:55 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.154.129@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:08:55.087549 14184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:55.078408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:39729 (local address 127.28.154.129:59607)
0811 09:08:55.078947 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:55.078962 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:55.079126 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:55.079494 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:55.079499 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:55.079914 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:08:55.080097 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:55.080109 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:55.081184 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:55.081189 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:55.082471 (+ 1282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:55.082489 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:55.083377 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:55.083383 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:55.083528 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:55.084197 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:08:55.084216 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:08:55.087276 (+ 3060us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.154.190:39729: Server oryx/127.28.154.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":245,"threads_started":1}
W20260811 09:08:55.087805 13684 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:08:55.078517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.129:59607 (local address 127.28.154.190:39729)
0811 09:08:55.078727 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:55.078734 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:55.079048 (+ 314us) server_negotiation.cc:407] Connection header received
0811 09:08:55.079271 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:55.079276 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:55.079353 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:55.079442 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:55.080261 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:55.081050 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:55.082660 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:55.083222 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:55.083342 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:55.087561 (+ 4219us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.154.129:59607: BlockingRecv error: recv got EOF from 127.28.154.129:59607 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 11 09:08:55 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.154.130@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:08:55 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.154.130@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:08:55.553632 14189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:55.544801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:39729 (local address 127.28.154.130:35261)
0811 09:08:55.545377 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:55.545392 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:55.545523 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:55.545873 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:55.545878 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:55.546350 (+ 472us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:08:55.546527 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:55.546545 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:55.547557 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:55.547561 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:55.548575 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:55.548588 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:55.549365 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:55.549370 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:55.549540 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:55.550178 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:08:55.550198 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:08:55.553278 (+ 3080us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.154.190:39729: Server oryx/127.28.154.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":294,"threads_started":1}
W20260811 09:08:55.553754 13684 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:08:55.545016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.130:35261 (local address 127.28.154.190:39729)
0811 09:08:55.545241 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:55.545247 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:55.545472 (+ 225us) server_negotiation.cc:407] Connection header received
0811 09:08:55.545678 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:55.545682 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:55.545746 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:55.545829 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:55.546670 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:55.547415 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:55.548713 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:55.549210 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:55.549338 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:55.553496 (+ 4158us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.154.130:35261: BlockingRecv error: recv got EOF from 127.28.154.130:35261 (error 108)
Metrics: {"server-negotiator.queue_time_us":102}
Aug 11 09:08:56 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.154.131@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:08:56 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.154.131@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:08:56.035820 14190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:56.027077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:39729 (local address 127.28.154.131:56001)
0811 09:08:56.027604 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:56.027620 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:56.027739 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:56.028091 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:56.028094 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:56.028495 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:08:56.028675 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:56.028685 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:56.029653 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:56.029656 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:56.030564 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:56.030578 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:56.031371 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:56.031377 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:56.031532 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:56.032219 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:08:56.032245 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:08:56.035564 (+ 3319us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.154.190:39729: Server oryx/127.28.154.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":253,"threads_started":1}
W20260811 09:08:56.036026 13684 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:08:56.027213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.131:56001 (local address 127.28.154.190:39729)
0811 09:08:56.027400 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:56.027404 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:56.027735 (+ 331us) server_negotiation.cc:407] Connection header received
0811 09:08:56.027842 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:56.027847 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:56.027931 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:56.028071 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:56.028815 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:56.029539 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:56.030737 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:56.031184 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:56.031294 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:56.035789 (+ 4495us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.154.131:56001: BlockingRecv error: recv got EOF from 127.28.154.131:56001 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 11 09:08:56 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.154.129@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:08:56 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.154.129@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:08:56.096716 14191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:56.088533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:39729 (local address 127.28.154.129:57447)
0811 09:08:56.089066 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:56.089084 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:56.089211 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:56.089500 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:56.089503 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:56.089913 (+ 410us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:08:56.090082 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:56.090092 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:56.091132 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:56.091136 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:56.092172 (+ 1036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:56.092193 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:56.093030 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:56.093037 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:56.093166 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:56.093785 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:08:56.093803 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:08:56.096472 (+ 2669us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.154.190:39729: Server oryx/127.28.154.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":292,"threads_started":1}
W20260811 09:08:56.096897 13684 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:08:56.088687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.129:57447 (local address 127.28.154.190:39729)
0811 09:08:56.088889 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:56.088894 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:56.089159 (+ 265us) server_negotiation.cc:407] Connection header received
0811 09:08:56.089326 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:56.089330 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:56.089394 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:56.089473 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:56.090209 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:56.090961 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:56.092320 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:56.092866 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:56.092961 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:56.096666 (+ 3705us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.154.129:57447: BlockingRecv error: recv got EOF from 127.28.154.129:57447 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 11 09:08:56 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.154.130@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:08:56 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.154.130@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:08:56.567317 14192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:56.554676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:39729 (local address 127.28.154.130:49347)
0811 09:08:56.555206 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:56.555222 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:56.555356 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:56.555690 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:56.555695 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:56.556099 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:08:56.556256 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:56.556266 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:56.557272 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:56.557277 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:56.558236 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:56.558271 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:56.559059 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:56.559065 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:56.559262 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:56.564444 (+ 5182us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:08:56.564466 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:08:56.567082 (+ 2616us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.154.190:39729: Server oryx/127.28.154.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":307,"threads_started":1}
W20260811 09:08:56.567626 13684 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:08:56.554784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.130:49347 (local address 127.28.154.190:39729)
0811 09:08:56.554971 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:56.554978 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:56.555308 (+ 330us) server_negotiation.cc:407] Connection header received
0811 09:08:56.555481 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:56.555485 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:56.555549 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:56.555640 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:56.556410 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:56.557142 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:56.558401 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:56.558872 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:56.558986 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:56.567370 (+ 8384us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.154.130:49347: BlockingRecv error: recv got EOF from 127.28.154.130:49347 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 11 09:08:57 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.154.131@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:08:57 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.154.131@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:08:57.045547 14194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:57.036763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:39729 (local address 127.28.154.131:39851)
0811 09:08:57.037345 (+ 582us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:57.037362 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:57.037487 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:57.037845 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:57.037854 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:57.038275 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:08:57.038443 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:57.038453 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:57.039531 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:57.039536 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:57.040585 (+ 1049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:57.040604 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:57.041405 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:57.041410 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:57.041550 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:57.042175 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:08:57.042199 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:08:57.045304 (+ 3105us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.154.190:39729: Server oryx/127.28.154.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":340,"threads_started":1}
W20260811 09:08:57.045845 13684 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:08:57.036888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.131:39851 (local address 127.28.154.190:39729)
0811 09:08:57.037048 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:57.037060 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:57.037436 (+ 376us) server_negotiation.cc:407] Connection header received
0811 09:08:57.037595 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:57.037599 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:57.037667 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:57.037759 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:57.038585 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:57.039370 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:57.040746 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:57.041216 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:57.041321 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:57.045565 (+ 4244us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.154.131:39851: BlockingRecv error: recv got EOF from 127.28.154.131:39851 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 11 09:08:57 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.154.129@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:08:57 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.154.129@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:08:57.106820 14195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:57.097779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:39729 (local address 127.28.154.129:41537)
0811 09:08:57.098327 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:57.098345 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:57.098482 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:57.098838 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:57.098842 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:57.099285 (+ 443us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:08:57.099460 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:57.099470 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:57.100589 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:57.100594 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:57.101564 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:57.101578 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:57.102458 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:57.102464 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:57.102594 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:57.103187 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:08:57.103207 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:08:57.106459 (+ 3252us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.154.190:39729: Server oryx/127.28.154.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":250,"threads_started":1}
W20260811 09:08:57.106930 13684 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:08:57.097890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.129:41537 (local address 127.28.154.190:39729)
0811 09:08:57.098151 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:57.098157 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:57.098457 (+ 300us) server_negotiation.cc:407] Connection header received
0811 09:08:57.098581 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:57.098584 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:57.098650 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:57.098756 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:57.099623 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:57.100420 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:57.101792 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:57.102242 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:57.102367 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:57.106696 (+ 4329us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.154.129:41537: BlockingRecv error: recv got EOF from 127.28.154.129:41537 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 11 09:08:57 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.154.130@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:08:57 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.154.130@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:08:57.577594 14196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:57.568378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:39729 (local address 127.28.154.130:38771)
0811 09:08:57.569015 (+ 637us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:57.569040 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:57.569167 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:57.569537 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:57.569541 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:57.569969 (+ 428us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:08:57.570164 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:57.570174 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:57.571438 (+ 1264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:57.571442 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:57.572471 (+ 1029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:57.572489 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:57.573214 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:57.573219 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:57.573344 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:57.574126 (+ 782us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:08:57.574154 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:08:57.577246 (+ 3092us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.154.190:39729: Server oryx/127.28.154.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":520,"thread_start_us":291,"threads_started":1}
W20260811 09:08:57.577715 13684 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:08:57.568548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.130:38771 (local address 127.28.154.190:39729)
0811 09:08:57.568767 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:57.568772 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:57.569146 (+ 374us) server_negotiation.cc:407] Connection header received
0811 09:08:57.569337 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:57.569342 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:57.569415 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:57.569507 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:57.570311 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:57.571271 (+ 960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:57.572641 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:57.573053 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:57.573152 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:57.577472 (+ 4320us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.154.130:38771: BlockingRecv error: recv got EOF from 127.28.154.130:38771 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
Aug 11 09:08:58 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.154.131@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:08:58 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.154.131@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:08:58.055011 14198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:58.046563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:39729 (local address 127.28.154.131:53807)
0811 09:08:58.047170 (+ 607us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:58.047186 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:58.047303 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:58.047648 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:58.047651 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:58.048074 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:08:58.048257 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:58.048266 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:58.049451 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:58.049455 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:58.050480 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:58.050495 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:58.051223 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:58.051229 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:58.051375 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:58.051944 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:08:58.051962 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:08:58.054767 (+ 2805us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.154.190:39729: Server oryx/127.28.154.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":250,"threads_started":1}
W20260811 09:08:58.055171 13684 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:08:58.046663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.131:53807 (local address 127.28.154.190:39729)
0811 09:08:58.046864 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:58.046869 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:58.047268 (+ 399us) server_negotiation.cc:407] Connection header received
0811 09:08:58.047432 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:58.047436 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:58.047506 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:58.047602 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:58.048405 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:58.049299 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:58.050636 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:58.051069 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:58.051180 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:58.054967 (+ 3787us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.154.131:53807: BlockingRecv error: recv got EOF from 127.28.154.131:53807 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Aug 11 09:08:58 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.154.129@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:08:58 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.154.129@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:08:58.116900 14202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:58.108067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:39729 (local address 127.28.154.129:35581)
0811 09:08:58.108649 (+ 582us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:58.108666 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:58.108789 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:58.109160 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:58.109164 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:58.109599 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:08:58.109756 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:58.109766 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:58.111035 (+ 1269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:58.111039 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:58.112145 (+ 1106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:58.112160 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:58.113221 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:58.113226 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:58.113346 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:58.113939 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:08:58.113957 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:08:58.116669 (+ 2712us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.154.190:39729: Server oryx/127.28.154.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":301,"threads_started":1}
W20260811 09:08:58.117168 13684 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:08:58.108295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.129:35581 (local address 127.28.154.190:39729)
0811 09:08:58.108497 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:58.108502 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:58.108766 (+ 264us) server_negotiation.cc:407] Connection header received
0811 09:08:58.108915 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:58.108920 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:58.108997 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:58.109100 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:58.109920 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:58.110879 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:58.112316 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:58.113055 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:58.113160 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:58.116899 (+ 3739us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.154.129:35581: BlockingRecv error: recv got EOF from 127.28.154.129:35581 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 11 09:08:58 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.154.130@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:08:58 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.154.130@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:08:58.589078 14205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:58.578581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:39729 (local address 127.28.154.130:53319)
0811 09:08:58.579192 (+ 611us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:58.579211 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:58.579378 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:58.579711 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:58.579716 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:58.580195 (+ 479us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:08:58.580389 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:58.580402 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:58.581560 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:58.581565 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:58.583526 (+ 1961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:58.583559 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:58.584782 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:58.584789 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:58.584946 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:58.585524 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:08:58.585542 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:08:58.588825 (+ 3283us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.154.190:39729: Server oryx/127.28.154.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":343,"threads_started":1}
W20260811 09:08:58.589221 13684 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:08:58.578714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.130:53319 (local address 127.28.154.190:39729)
0811 09:08:58.578980 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:58.578987 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:58.579294 (+ 307us) server_negotiation.cc:407] Connection header received
0811 09:08:58.579507 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:58.579512 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:58.579580 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:58.579664 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:58.580529 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:58.581403 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:58.583768 (+ 2365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:58.584601 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:58.584728 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:58.589022 (+ 4294us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.154.130:53319: BlockingRecv error: recv got EOF from 127.28.154.130:53319 (error 108)
Metrics: {"server-negotiator.queue_time_us":100}
Aug 11 09:08:59 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.154.131@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:08:59 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.154.131@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:08:59.065099 14206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:59.056099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:39729 (local address 127.28.154.131:43983)
0811 09:08:59.056755 (+ 656us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:59.056772 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:59.056922 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:59.057270 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:59.057275 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:59.057797 (+ 522us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:08:59.058008 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:59.058021 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:59.059291 (+ 1270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:59.059295 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:59.060436 (+ 1141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:59.060451 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:59.061310 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:59.061317 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:59.061490 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:59.062063 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:08:59.062080 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:08:59.064862 (+ 2782us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.154.190:39729: Server oryx/127.28.154.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":312,"threads_started":1}
W20260811 09:08:59.065377 13684 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:08:59.056243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.131:43983 (local address 127.28.154.190:39729)
0811 09:08:59.056508 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:59.056514 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:59.056859 (+ 345us) server_negotiation.cc:407] Connection header received
0811 09:08:59.057048 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:59.057053 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:59.057125 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:59.057212 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:59.058164 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:59.059150 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:59.060571 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:59.061132 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:59.061230 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:59.065105 (+ 3875us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.154.131:43983: BlockingRecv error: recv got EOF from 127.28.154.131:43983 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Aug 11 09:08:59 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.154.129@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:08:59 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.154.129@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:08:59.127243 14207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:59.117980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:39729 (local address 127.28.154.129:48745)
0811 09:08:59.118547 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:59.118564 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:59.118706 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:59.119051 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:59.119055 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:59.119516 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:08:59.119728 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:59.119744 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:59.120947 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:59.120951 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:59.121939 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:59.121953 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:59.123050 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:59.123057 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:59.123196 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:59.123853 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:08:59.123906 (+ 53us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:08:59.126982 (+ 3076us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.154.190:39729: Server oryx/127.28.154.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":275,"threads_started":1}
W20260811 09:08:59.127416 13684 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:08:59.118094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.129:48745 (local address 127.28.154.190:39729)
0811 09:08:59.118338 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:59.118344 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:59.118650 (+ 306us) server_negotiation.cc:407] Connection header received
0811 09:08:59.118828 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:59.118832 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:59.118914 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:59.119001 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:59.119892 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:59.120805 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:59.122103 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:59.122764 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:59.122887 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:59.127207 (+ 4320us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.154.129:48745: BlockingRecv error: recv got EOF from 127.28.154.129:48745 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
Aug 11 09:08:59 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.154.130@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:08:59 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.154.130@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:08:59.598585 14209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:08:59.590053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:39729 (local address 127.28.154.130:49945)
0811 09:08:59.590581 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0811 09:08:59.590598 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 09:08:59.590755 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:08:59.591089 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:08:59.591094 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:08:59.591514 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:08:59.591683 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:59.591693 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:59.592750 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:59.592754 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:59.593714 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:08:59.593726 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:08:59.594458 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:08:59.594464 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:08:59.594631 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:59.595266 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:08:59.595285 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:08:59.598335 (+ 3050us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.154.190:39729: Server oryx/127.28.154.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":278,"threads_started":1}
W20260811 09:08:59.598786 13684 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:08:59.590135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.130:49945 (local address 127.28.154.190:39729)
0811 09:08:59.590325 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 09:08:59.590331 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:08:59.590695 (+ 364us) server_negotiation.cc:407] Connection header received
0811 09:08:59.590883 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:08:59.590887 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:08:59.590949 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:08:59.591030 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:08:59.591830 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:59.592613 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:59.593857 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:08:59.594304 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:08:59.594420 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:08:59.598528 (+ 4108us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.154.130:49945: BlockingRecv error: recv got EOF from 127.28.154.130:49945 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 11 09:09:00 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.154.131@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:09:00 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.154.131@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:09:00.074695 14211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:09:00.066016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:39729 (local address 127.28.154.131:58947)
0811 09:09:00.066564 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0811 09:09:00.066581 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 09:09:00.066696 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:09:00.067007 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:09:00.067011 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:09:00.067380 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:09:00.067552 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:09:00.067561 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:09:00.068648 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:09:00.068652 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:09:00.069840 (+ 1188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:09:00.069856 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:09:00.070617 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:09:00.070622 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:09:00.070976 (+ 354us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:09:00.071588 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:09:00.071607 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:09:00.074428 (+ 2821us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.154.190:39729: Server oryx/127.28.154.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":276,"threads_started":1}
W20260811 09:09:00.074951 13684 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:09:00.066097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.131:58947 (local address 127.28.154.190:39729)
0811 09:09:00.066252 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 09:09:00.066256 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:09:00.066664 (+ 408us) server_negotiation.cc:407] Connection header received
0811 09:09:00.066789 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:09:00.066793 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:09:00.066858 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:09:00.066953 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:09:00.067716 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:09:00.068540 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:09:00.070006 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:09:00.070469 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:09:00.070577 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:09:00.074690 (+ 4113us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.154.131:58947: BlockingRecv error: recv got EOF from 127.28.154.131:58947 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 11 09:09:00 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.154.129@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:09:00 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.154.129@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:09:00.136806 14212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:09:00.128281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:39729 (local address 127.28.154.129:51105)
0811 09:09:00.128832 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0811 09:09:00.128848 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 09:09:00.128988 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:09:00.129339 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:09:00.129344 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:09:00.129750 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:09:00.129907 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:09:00.129917 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:09:00.130961 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:09:00.130966 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:09:00.131897 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:09:00.131911 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:09:00.132662 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:09:00.132668 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:09:00.132830 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:09:00.133440 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:09:00.133458 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:09:00.136523 (+ 3065us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.154.190:39729: Server oryx/127.28.154.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":302,"threads_started":1}
W20260811 09:09:00.137176 13684 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:09:00.128440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.129:51105 (local address 127.28.154.190:39729)
0811 09:09:00.128659 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0811 09:09:00.128666 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:09:00.128935 (+ 269us) server_negotiation.cc:407] Connection header received
0811 09:09:00.129140 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:09:00.129144 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:09:00.129210 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:09:00.129295 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:09:00.130057 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:09:00.130825 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:09:00.132038 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:09:00.132511 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:09:00.132630 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:09:00.136891 (+ 4261us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.154.129:51105: BlockingRecv error: recv got EOF from 127.28.154.129:51105 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 11 09:09:00 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.154.130@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:09:00 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.154.130@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:09:00.608368 14213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:09:00.599609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:39729 (local address 127.28.154.130:46217)
0811 09:09:00.600296 (+ 687us) negotiation.cc:107] Waiting for socket to connect
0811 09:09:00.600314 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 09:09:00.600451 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:09:00.600834 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:09:00.600839 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:09:00.601257 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:09:00.601409 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:09:00.601418 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:09:00.602430 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:09:00.602435 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:09:00.603425 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:09:00.603439 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:09:00.604161 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:09:00.604166 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:09:00.604294 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:09:00.605024 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:09:00.605051 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:09:00.608117 (+ 3066us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.154.190:39729: Server oryx/127.28.154.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":569,"thread_start_us":450,"threads_started":1}
W20260811 09:09:00.608595 13684 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:09:00.599719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.130:46217 (local address 127.28.154.190:39729)
0811 09:09:00.599942 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0811 09:09:00.599949 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:09:00.600405 (+ 456us) server_negotiation.cc:407] Connection header received
0811 09:09:00.600602 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:09:00.600606 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:09:00.600674 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:09:00.600755 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:09:00.601542 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:09:00.602282 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:09:00.603575 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:09:00.604020 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:09:00.604118 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:09:00.608305 (+ 4187us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.154.130:46217: BlockingRecv error: recv got EOF from 127.28.154.130:46217 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 11 09:09:01 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.154.131@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:09:01 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.154.131@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:09:01.084122 14214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:09:01.075684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:39729 (local address 127.28.154.131:58015)
0811 09:09:01.076220 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0811 09:09:01.076235 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 09:09:01.076388 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:09:01.076728 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:09:01.076731 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:09:01.077221 (+ 490us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:09:01.077376 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:09:01.077386 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:09:01.078625 (+ 1239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:09:01.078630 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:09:01.079665 (+ 1035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:09:01.079678 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:09:01.080324 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:09:01.080328 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:09:01.080485 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:09:01.081039 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:09:01.081057 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:09:01.083858 (+ 2801us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.154.190:39729: Server oryx/127.28.154.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":286,"threads_started":1}
W20260811 09:09:01.084319 13684 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:09:01.075884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.131:58015 (local address 127.28.154.190:39729)
0811 09:09:01.076116 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0811 09:09:01.076122 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:09:01.076311 (+ 189us) server_negotiation.cc:407] Connection header received
0811 09:09:01.076519 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:09:01.076523 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:09:01.076587 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:09:01.076689 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:09:01.077525 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:09:01.078511 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:09:01.079782 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:09:01.080211 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:09:01.080304 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:09:01.084067 (+ 3763us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.154.131:58015: BlockingRecv error: recv got EOF from 127.28.154.131:58015 (error 108)
Metrics: {"server-negotiator.queue_time_us":95}
Aug 11 09:09:01 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.154.129@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:09:01 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.154.129@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:09:01.146512 14215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:09:01.137838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:39729 (local address 127.28.154.129:55171)
0811 09:09:01.138364 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0811 09:09:01.138381 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 09:09:01.138520 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:09:01.138903 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:09:01.138909 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:09:01.139471 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:09:01.139667 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:09:01.139680 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:09:01.140738 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:09:01.140742 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:09:01.141670 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:09:01.141685 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:09:01.142424 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:09:01.142432 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:09:01.142616 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:09:01.143184 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:09:01.143203 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:09:01.146178 (+ 2975us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.154.190:39729: Server oryx/127.28.154.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":265,"threads_started":1}
W20260811 09:09:01.146692 13684 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:09:01.137929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.129:55171 (local address 127.28.154.190:39729)
0811 09:09:01.138129 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 09:09:01.138135 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:09:01.138459 (+ 324us) server_negotiation.cc:407] Connection header received
0811 09:09:01.138647 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:09:01.138651 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:09:01.138726 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:09:01.138825 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:09:01.139815 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:09:01.140587 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:09:01.141794 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:09:01.142212 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:09:01.142315 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:09:01.146459 (+ 4144us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.154.129:55171: BlockingRecv error: recv got EOF from 127.28.154.129:55171 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 11 09:09:01 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.154.130@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:09:01 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.154.130@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:09:01.620473 14216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:09:01.609313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:39729 (local address 127.28.154.130:47721)
0811 09:09:01.609805 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0811 09:09:01.609822 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 09:09:01.609937 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:09:01.610224 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:09:01.610227 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:09:01.610667 (+ 440us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:09:01.610864 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:09:01.610877 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:09:01.611890 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:09:01.611894 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:09:01.612827 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:09:01.612839 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:09:01.613535 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:09:01.613539 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:09:01.613662 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:09:01.617005 (+ 3343us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:09:01.617023 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:09:01.620147 (+ 3124us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.154.190:39729: Server oryx/127.28.154.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":253,"threads_started":1}
W20260811 09:09:01.620690 13684 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:09:01.609420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.130:47721 (local address 127.28.154.190:39729)
0811 09:09:01.609583 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 09:09:01.609587 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:09:01.609889 (+ 302us) server_negotiation.cc:407] Connection header received
0811 09:09:01.610030 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:09:01.610039 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:09:01.610104 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:09:01.610191 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:09:01.611025 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:09:01.611767 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:09:01.612958 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:09:01.613391 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:09:01.613487 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:09:01.620447 (+ 6960us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.154.130:47721: BlockingRecv error: recv got EOF from 127.28.154.130:47721 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 11 09:09:02 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.154.131@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:09:02 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.154.131@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:09:02.093854 14217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:09:02.085043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:39729 (local address 127.28.154.131:56263)
0811 09:09:02.085594 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0811 09:09:02.085611 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 09:09:02.085731 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:09:02.086122 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:09:02.086127 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:09:02.086578 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:09:02.086762 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:09:02.086772 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:09:02.087835 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:09:02.087840 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:09:02.088837 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:09:02.088853 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:09:02.089619 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:09:02.089625 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:09:02.089761 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:09:02.090432 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:09:02.090452 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:09:02.093543 (+ 3091us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.154.190:39729: Server oryx/127.28.154.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":288,"threads_started":1}
W20260811 09:09:02.094228 13684 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:09:02.085129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.131:56263 (local address 127.28.154.190:39729)
0811 09:09:02.085319 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 09:09:02.085324 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:09:02.085703 (+ 379us) server_negotiation.cc:407] Connection header received
0811 09:09:02.085875 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:09:02.085879 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:09:02.085948 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:09:02.086039 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:09:02.086947 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:09:02.087700 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:09:02.088997 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:09:02.089426 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:09:02.089565 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:09:02.093921 (+ 4356us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.154.131:56263: BlockingRecv error: recv got EOF from 127.28.154.131:56263 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 11 09:09:02 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.154.129@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:09:02 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.154.129@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:09:02.156585 14221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:09:02.147465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:39729 (local address 127.28.154.129:60173)
0811 09:09:02.148070 (+ 605us) negotiation.cc:107] Waiting for socket to connect
0811 09:09:02.148087 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 09:09:02.148241 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:09:02.148639 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:09:02.148644 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:09:02.149076 (+ 432us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:09:02.149236 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:09:02.149246 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:09:02.150374 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:09:02.150379 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:09:02.151354 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:09:02.151369 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:09:02.152438 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:09:02.152443 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:09:02.152620 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:09:02.153302 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:09:02.153323 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:09:02.156250 (+ 2927us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.154.190:39729: Server oryx/127.28.154.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":390,"threads_started":1}
W20260811 09:09:02.156812 13684 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:09:02.147556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.129:60173 (local address 127.28.154.190:39729)
0811 09:09:02.147729 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 09:09:02.147733 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:09:02.148160 (+ 427us) server_negotiation.cc:407] Connection header received
0811 09:09:02.148352 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:09:02.148355 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:09:02.148456 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:09:02.148557 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:09:02.149388 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:09:02.150227 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:09:02.151550 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:09:02.152217 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:09:02.152379 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:09:02.156543 (+ 4164us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.154.129:60173: BlockingRecv error: recv got EOF from 127.28.154.129:60173 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 11 09:09:02 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.154.130@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:09:02 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.154.130@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:09:02.630069 14223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:09:02.621448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:39729 (local address 127.28.154.130:44679)
0811 09:09:02.621975 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0811 09:09:02.621991 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 09:09:02.622137 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:09:02.622473 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:09:02.622478 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:09:02.622913 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:09:02.623076 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:09:02.623085 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:09:02.624084 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:09:02.624088 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:09:02.625041 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:09:02.625053 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:09:02.625781 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:09:02.625787 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:09:02.625963 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:09:02.626564 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:09:02.626581 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:09:02.629828 (+ 3247us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.154.190:39729: Server oryx/127.28.154.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":295,"threads_started":1}
W20260811 09:09:02.630229 13684 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:09:02.621609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.130:44679 (local address 127.28.154.190:39729)
0811 09:09:02.621852 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0811 09:09:02.621858 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:09:02.622078 (+ 220us) server_negotiation.cc:407] Connection header received
0811 09:09:02.622268 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:09:02.622271 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:09:02.622332 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:09:02.622423 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:09:02.623211 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:09:02.623930 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:09:02.625177 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:09:02.625633 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:09:02.625752 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:09:02.630036 (+ 4284us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.154.130:44679: BlockingRecv error: recv got EOF from 127.28.154.130:44679 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
Aug 11 09:09:03 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.154.131@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:09:03 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.154.131@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:09:03.105249 14224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:09:03.095062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:39729 (local address 127.28.154.131:35727)
0811 09:09:03.095592 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0811 09:09:03.095613 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 09:09:03.095834 (+ 221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:09:03.096319 (+ 485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:09:03.096325 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:09:03.096863 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:09:03.097091 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:09:03.097113 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:09:03.098859 (+ 1746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:09:03.098864 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:09:03.099851 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:09:03.099866 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:09:03.100749 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:09:03.100754 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:09:03.100904 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:09:03.101618 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:09:03.101640 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:09:03.104987 (+ 3347us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.154.190:39729: Server oryx/127.28.154.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":416,"thread_start_us":299,"threads_started":1}
W20260811 09:09:03.105783 13684 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:09:03.095227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.131:35727 (local address 127.28.154.190:39729)
0811 09:09:03.095406 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 09:09:03.095411 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:09:03.095735 (+ 324us) server_negotiation.cc:407] Connection header received
0811 09:09:03.096044 (+ 309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:09:03.096049 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:09:03.096138 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:09:03.096257 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:09:03.097281 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:09:03.098592 (+ 1311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:09:03.100040 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:09:03.100566 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:09:03.100688 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:09:03.105320 (+ 4632us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.154.131:35727: BlockingRecv error: recv got EOF from 127.28.154.131:35727 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 11 09:09:03 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.154.129@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:09:03 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.154.129@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:09:03.166394 14226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:09:03.157725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:39729 (local address 127.28.154.129:51443)
0811 09:09:03.158311 (+ 586us) negotiation.cc:107] Waiting for socket to connect
0811 09:09:03.158326 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 09:09:03.158470 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:09:03.158834 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:09:03.158839 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:09:03.159285 (+ 446us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:09:03.159457 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:09:03.159469 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:09:03.160596 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:09:03.160602 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:09:03.161916 (+ 1314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:09:03.161933 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:09:03.162700 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:09:03.162705 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:09:03.162827 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:09:03.163408 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:09:03.163426 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:09:03.166145 (+ 2719us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.154.190:39729: Server oryx/127.28.154.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":314,"threads_started":1}
W20260811 09:09:03.166731 13684 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:09:03.157857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.129:51443 (local address 127.28.154.190:39729)
0811 09:09:03.158112 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0811 09:09:03.158118 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:09:03.158439 (+ 321us) server_negotiation.cc:407] Connection header received
0811 09:09:03.158602 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:09:03.158606 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:09:03.158687 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:09:03.158776 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:09:03.159587 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:09:03.160425 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:09:03.162066 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:09:03.162522 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:09:03.162653 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:09:03.166357 (+ 3704us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.154.129:51443: BlockingRecv error: recv got EOF from 127.28.154.129:51443 (error 108)
Metrics: {"server-negotiator.queue_time_us":105}
Aug 11 09:09:03 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.154.130@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:09:03 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.154.130@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:09:03.640040 14227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:09:03.631236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:39729 (local address 127.28.154.130:59847)
0811 09:09:03.632069 (+ 833us) negotiation.cc:107] Waiting for socket to connect
0811 09:09:03.632085 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 09:09:03.632222 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:09:03.632615 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:09:03.632619 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:09:03.633089 (+ 470us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:09:03.633296 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:09:03.633310 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:09:03.634423 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:09:03.634427 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:09:03.635417 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:09:03.635430 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:09:03.636214 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:09:03.636219 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:09:03.636366 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:09:03.636915 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:09:03.636933 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:09:03.639776 (+ 2843us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.154.190:39729: Server oryx/127.28.154.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":722,"thread_start_us":341,"threads_started":1}
W20260811 09:09:03.640228 13684 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:09:03.631555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.130:59847 (local address 127.28.154.190:39729)
0811 09:09:03.631752 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0811 09:09:03.631758 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:09:03.632199 (+ 441us) server_negotiation.cc:407] Connection header received
0811 09:09:03.632363 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:09:03.632368 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:09:03.632456 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:09:03.632559 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:09:03.633461 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:09:03.634265 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:09:03.635552 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:09:03.636051 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:09:03.636164 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:09:03.640005 (+ 3841us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.154.130:59847: BlockingRecv error: recv got EOF from 127.28.154.130:59847 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Aug 11 09:09:04 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.154.131@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:09:04 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.154.131@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:09:04.115999 14228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:09:04.106648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:39729 (local address 127.28.154.131:59821)
0811 09:09:04.107376 (+ 728us) negotiation.cc:107] Waiting for socket to connect
0811 09:09:04.107394 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 09:09:04.107557 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:09:04.107900 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:09:04.107905 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:09:04.108414 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:09:04.108633 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:09:04.108643 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:09:04.109855 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:09:04.109860 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:09:04.111042 (+ 1182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:09:04.111056 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:09:04.111836 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:09:04.111840 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:09:04.112015 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:09:04.112795 (+ 780us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:09:04.112814 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:09:04.115766 (+ 2952us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.154.190:39729: Server oryx/127.28.154.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":620,"thread_start_us":321,"threads_started":1}
W20260811 09:09:04.116257 13684 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:09:04.106848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.131:59821 (local address 127.28.154.190:39729)
0811 09:09:04.107091 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0811 09:09:04.107098 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:09:04.107506 (+ 408us) server_negotiation.cc:407] Connection header received
0811 09:09:04.107664 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:09:04.107669 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:09:04.107750 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:09:04.107842 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:09:04.108900 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:09:04.109716 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:09:04.111176 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:09:04.111637 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:09:04.111779 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:09:04.116013 (+ 4234us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.154.131:59821: BlockingRecv error: recv got EOF from 127.28.154.131:59821 (error 108)
Metrics: {"server-negotiator.queue_time_us":96}
Aug 11 09:09:04 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.154.129@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:09:04 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.154.129@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:09:04.176160 14229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:09:04.167371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:39729 (local address 127.28.154.129:55349)
0811 09:09:04.167920 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0811 09:09:04.167942 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 09:09:04.168086 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:09:04.168472 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:09:04.168477 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:09:04.168890 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:09:04.169079 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:09:04.169090 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:09:04.170190 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:09:04.170195 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:09:04.171124 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:09:04.171138 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:09:04.171831 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:09:04.171838 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:09:04.172005 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:09:04.172674 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:09:04.172692 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:09:04.175787 (+ 3095us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.154.190:39729: Server oryx/127.28.154.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":277,"threads_started":1}
W20260811 09:09:04.176293 13684 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:09:04.167556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.129:55349 (local address 127.28.154.190:39729)
0811 09:09:04.167922 (+ 366us) server_negotiation.cc:206] Beginning negotiation
0811 09:09:04.167929 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:09:04.168023 (+ 94us) server_negotiation.cc:407] Connection header received
0811 09:09:04.168204 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:09:04.168208 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:09:04.168289 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:09:04.168426 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:09:04.169241 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:09:04.170041 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:09:04.171268 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:09:04.171689 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:09:04.171798 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:09:04.176046 (+ 4248us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.154.129:55349: BlockingRecv error: recv got EOF from 127.28.154.129:55349 (error 108)
Metrics: {"server-negotiator.queue_time_us":241}
Aug 11 09:09:04 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.154.130@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:09:04 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.154.130@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:09:04.650194 14236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:09:04.641099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:39729 (local address 127.28.154.130:57905)
0811 09:09:04.641603 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0811 09:09:04.641620 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 09:09:04.641749 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:09:04.642156 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:09:04.642160 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:09:04.642542 (+ 382us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:09:04.642770 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:09:04.642780 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:09:04.643889 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:09:04.643894 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:09:04.644869 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:09:04.644881 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:09:04.645647 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:09:04.645654 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:09:04.645936 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:09:04.646613 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:09:04.646638 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:09:04.649913 (+ 3275us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.154.190:39729: Server oryx/127.28.154.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":262,"threads_started":1}
W20260811 09:09:04.650341 13684 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:09:04.641185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.130:57905 (local address 127.28.154.190:39729)
0811 09:09:04.641431 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0811 09:09:04.641437 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:09:04.641684 (+ 247us) server_negotiation.cc:407] Connection header received
0811 09:09:04.641892 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:09:04.641898 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:09:04.642008 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:09:04.642113 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:09:04.642947 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:09:04.643740 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:09:04.645008 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:09:04.645478 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:09:04.645586 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:09:04.650158 (+ 4572us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.154.130:57905: BlockingRecv error: recv got EOF from 127.28.154.130:57905 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Aug 11 09:09:05 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.154.131@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:09:05 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.154.131@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:09:05.134259 14237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:09:05.116983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:39729 (local address 127.28.154.131:58437)
0811 09:09:05.117567 (+ 584us) negotiation.cc:107] Waiting for socket to connect
0811 09:09:05.117588 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 09:09:05.117732 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:09:05.118071 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:09:05.118076 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:09:05.118506 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:09:05.118690 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:09:05.118700 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:09:05.119724 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:09:05.119729 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:09:05.120874 (+ 1145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:09:05.120889 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:09:05.121727 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:09:05.121731 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:09:05.121856 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:09:05.130798 (+ 8942us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:09:05.130821 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:09:05.134007 (+ 3186us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.154.190:39729: Server oryx/127.28.154.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":290,"threads_started":1}
W20260811 09:09:05.134569 13684 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:09:05.117147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.131:58437 (local address 127.28.154.190:39729)
0811 09:09:05.117396 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0811 09:09:05.117402 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:09:05.117673 (+ 271us) server_negotiation.cc:407] Connection header received
0811 09:09:05.117856 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:09:05.117861 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:09:05.117927 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:09:05.118034 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:09:05.118834 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:09:05.119578 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:09:05.121031 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:09:05.121588 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:09:05.121687 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:09:05.134282 (+ 12595us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.154.131:58437: BlockingRecv error: recv got EOF from 127.28.154.131:58437 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Aug 11 09:09:05 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.154.129@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:09:05 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.154.129@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:09:05.186554 14238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:09:05.177609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:39729 (local address 127.28.154.129:42395)
0811 09:09:05.178190 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0811 09:09:05.178211 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 09:09:05.178348 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:09:05.178823 (+ 475us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:09:05.178827 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:09:05.179247 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:09:05.179419 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:09:05.179429 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:09:05.180564 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:09:05.180569 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:09:05.181554 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:09:05.181568 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:09:05.182520 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:09:05.182527 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:09:05.182718 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:09:05.183392 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:09:05.183440 (+ 48us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:09:05.186286 (+ 2846us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.154.190:39729: Server oryx/127.28.154.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":346,"threads_started":1}
W20260811 09:09:05.186702 13684 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:09:05.177782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.129:42395 (local address 127.28.154.190:39729)
0811 09:09:05.178011 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0811 09:09:05.178018 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:09:05.178297 (+ 279us) server_negotiation.cc:407] Connection header received
0811 09:09:05.178554 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:09:05.178558 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:09:05.178641 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:09:05.178733 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:09:05.179660 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:09:05.180431 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:09:05.181710 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:09:05.182348 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:09:05.182476 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:09:05.186491 (+ 4015us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.154.129:42395: BlockingRecv error: recv got EOF from 127.28.154.129:42395 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
Aug 11 09:09:05 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.154.130@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 09:09:05 dist-test-slave-xt4k krb5kdc[12376](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.154.130@KRBTEST.COM for oryx/127.28.154.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 09:09:05.659761 14241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:09:05.651320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:39729 (local address 127.28.154.130:42903)
0811 09:09:05.651869 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0811 09:09:05.651886 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 09:09:05.652033 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:09:05.652447 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:09:05.652452 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:09:05.652863 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:09:05.653032 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:09:05.653042 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:09:05.654071 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:09:05.654076 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:09:05.655017 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:09:05.655030 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:09:05.655785 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:09:05.655791 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:09:05.655969 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:09:05.656555 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:09:05.656573 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:09:05.659431 (+ 2858us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.154.190:39729: Server oryx/127.28.154.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":297,"threads_started":1}
W20260811 09:09:05.659822 13684 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:09:05.651413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.130:42903 (local address 127.28.154.190:39729)
0811 09:09:05.651661 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0811 09:09:05.651668 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:09:05.651984 (+ 316us) server_negotiation.cc:407] Connection header received
0811 09:09:05.652157 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:09:05.652163 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:09:05.652233 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:09:05.652315 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:09:05.653179 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:09:05.653925 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:09:05.655158 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:09:05.655641 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:09:05.655756 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:09:05.659599 (+ 3843us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.154.130:42903: BlockingRecv error: recv got EOF from 127.28.154.130:42903 (error 108)
Metrics: {"server-negotiator.queue_time_us":102}
I20260811 09:09:05.957418 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 13413
I20260811 09:09:05.970105 14243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:09:05.958697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49972 (local address 127.28.154.129:41847)
0811 09:09:05.959255 (+ 558us) server_negotiation.cc:206] Beginning negotiation
0811 09:09:05.959263 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:09:05.959302 (+ 39us) server_negotiation.cc:407] Connection header received
0811 09:09:05.959394 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:09:05.959400 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:09:05.959631 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:09:05.959824 (+ 193us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:09:05.961366 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:09:05.962507 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:09:05.963680 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:09:05.964238 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:09:05.964374 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:09:05.966978 (+ 2604us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:09:05.966989 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:09:05.967005 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:09:05.967039 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:09:05.969557 (+ 2518us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.28.154.129@KRBTEST.COM not found in keytab (ticket kvno 2)
0811 09:09:05.969849 (+ 292us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:49972: Request ticket server kudu/127.28.154.129@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":273,"threads_started":1}
W20260811 09:09:05.970214 14243 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:49972: Request ticket server kudu/127.28.154.129@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260811 09:09:05.970392 29290 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 13413: Not authorized: Client connection negotiation failed: client connection to 127.28.154.129:41847: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.28.154.129@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260811 09:09:05.970595 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 13413
I20260811 09:09:05.999537 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 13686
I20260811 09:09:06.013439 14245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:09:06.000837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47040 (local address 127.28.154.130:36935)
0811 09:09:06.001558 (+ 721us) server_negotiation.cc:206] Beginning negotiation
0811 09:09:06.001605 (+ 47us) server_negotiation.cc:399] Waiting for connection header
0811 09:09:06.001638 (+ 33us) server_negotiation.cc:407] Connection header received
0811 09:09:06.001703 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:09:06.001709 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:09:06.001939 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:09:06.002095 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:09:06.003493 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:09:06.004729 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:09:06.006068 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:09:06.006718 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:09:06.006828 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:09:06.009522 (+ 2694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:09:06.009532 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:09:06.009550 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:09:06.009595 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:09:06.012859 (+ 3264us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.28.154.130@KRBTEST.COM not found in keytab (ticket kvno 2)
0811 09:09:06.013179 (+ 320us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:47040: Request ticket server kudu/127.28.154.130@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":560,"thread_start_us":360,"threads_started":1}
W20260811 09:09:06.013572 14245 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:47040: Request ticket server kudu/127.28.154.130@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260811 09:09:06.013640 29290 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 13686: Not authorized: Client connection negotiation failed: client connection to 127.28.154.130:36935: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.28.154.130@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260811 09:09:06.013827 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 13686
I20260811 09:09:06.037772 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 13912
I20260811 09:09:06.049882 14248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:09:06.038843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39178 (local address 127.28.154.131:32793)
0811 09:09:06.039410 (+ 567us) server_negotiation.cc:206] Beginning negotiation
0811 09:09:06.039427 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 09:09:06.039455 (+ 28us) server_negotiation.cc:407] Connection header received
0811 09:09:06.039527 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:09:06.039534 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:09:06.039782 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:09:06.039949 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:09:06.041285 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:09:06.042272 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:09:06.043463 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:09:06.044022 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:09:06.044122 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:09:06.046816 (+ 2694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:09:06.046824 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:09:06.046840 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:09:06.046876 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:09:06.049337 (+ 2461us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.28.154.131@KRBTEST.COM not found in keytab (ticket kvno 2)
0811 09:09:06.049629 (+ 292us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:39178: Request ticket server kudu/127.28.154.131@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":422,"thread_start_us":278,"threads_started":1}
W20260811 09:09:06.049998 14248 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:39178: Request ticket server kudu/127.28.154.131@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260811 09:09:06.050120 29290 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 13912: Not authorized: Client connection negotiation failed: client connection to 127.28.154.131:32793: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.28.154.131@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260811 09:09:06.050318 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 13912
I20260811 09:09:06.083722 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 13283
I20260811 09:09:06.085259 13402 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:09:06.208216 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 13283
2026-08-11T09:09:06Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (21762 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-11T09:09:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:09:06Z Disabled control of system clock
I20260811 09:09:06.307113 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439250192320-29290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439250192320-29290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439250192320-29290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.154.190:46867
--webserver_interface=127.28.154.190
--webserver_port=0
--builtin_ntp_servers=127.28.154.148:34091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.154.190:46867
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260811 09:09:06.669117 14258 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:09:06.669413 14258 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:09:06.669580 14258 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:09:06.676860 14258 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:09:06.677014 14258 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:09:06.677083 14258 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:09:06.677171 14258 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:09:06.677240 14258 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:09:06.686545 14258 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:34091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439250192320-29290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439250192320-29290-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.154.190:46867
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.154.190:46867
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.154.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439250192320-29290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:09:06.687816 14258 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:09:06.689612 14258 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:09:06.701884 14267 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:09:06.701972 14275 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:09:06.701906 14268 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:09:06.702725 14258 server_base.cc:1034] running on GCE node
I20260811 09:09:06.703604 14258 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:09:06.705183 14258 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:09:06.706389 14258 hybrid_clock.cc:648] HybridClock initialized: now 1786439346706339 us; error 39 us; skew 500 ppm
I20260811 09:09:06.706785 14258 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 09:09:06.713338 14258 webserver.cc:460] Webserver started at http://127.28.154.190:34167/ using document root <none> and password file <none>
I20260811 09:09:06.714144 14258 fs_manager.cc:359] Metadata directory not provided
I20260811 09:09:06.714238 14258 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:09:06.714592 14258 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:09:06.717635 14258 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439250192320-29290-0/minicluster-data/master-0/data/instance:
uuid: "f22db52014084600a5e7fcc973232576"
format_stamp: "Formatted at 2026-08-11 09:09:06 on dist-test-slave-xt4k"
server_key: "defd383e836215128b792ecfbcbc6709"
server_key_iv: "14172485f185184bb9ba7187413f2dfe"
server_key_version: "encryptionkey@0"
I20260811 09:09:06.718426 14258 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439250192320-29290-0/minicluster-data/master-0/wal/instance:
uuid: "f22db52014084600a5e7fcc973232576"
format_stamp: "Formatted at 2026-08-11 09:09:06 on dist-test-slave-xt4k"
server_key: "defd383e836215128b792ecfbcbc6709"
server_key_iv: "14172485f185184bb9ba7187413f2dfe"
server_key_version: "encryptionkey@0"
I20260811 09:09:06.724136 14258 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 09:09:06.727738 14285 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:09:06.729172 14258 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 09:09:06.729344 14258 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439250192320-29290-0/minicluster-data/master-0/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439250192320-29290-0/minicluster-data/master-0/wal
uuid: "f22db52014084600a5e7fcc973232576"
format_stamp: "Formatted at 2026-08-11 09:09:06 on dist-test-slave-xt4k"
server_key: "defd383e836215128b792ecfbcbc6709"
server_key_iv: "14172485f185184bb9ba7187413f2dfe"
server_key_version: "encryptionkey@0"
I20260811 09:09:06.729580 14258 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439250192320-29290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439250192320-29290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439250192320-29290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:09:06.757246 14258 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:09:06.758116 14258 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:09:06.758378 14258 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:09:06.775199 14258 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.190:46867
I20260811 09:09:06.775218 14387 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.190:46867 every 8 connection(s)
I20260811 09:09:06.776566 14258 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
I20260811 09:09:06.780650 14388 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:09:06.782951 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 14258
I20260811 09:09:06.783149 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439250192320-29290-0/minicluster-data/master-0/wal/instance
I20260811 09:09:06.783813 29290 external_mini_cluster.cc:1428] Setting key f4d71214a9483f38a15304e596964d23
I20260811 09:09:06.794735 14388 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f22db52014084600a5e7fcc973232576: Bootstrap starting.
I20260811 09:09:06.798869 14388 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f22db52014084600a5e7fcc973232576: Neither blocks nor log segments found. Creating new log.
I20260811 09:09:06.800272 14388 log.cc:824] T 00000000000000000000000000000000 P f22db52014084600a5e7fcc973232576: Log is configured to *not* fsync() on all Append() calls
I20260811 09:09:06.800686 14391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:09:06.785962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33946 (local address 127.28.154.190:46867)
0811 09:09:06.787119 (+ 1157us) server_negotiation.cc:206] Beginning negotiation
0811 09:09:06.787134 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 09:09:06.787189 (+ 55us) server_negotiation.cc:407] Connection header received
0811 09:09:06.789490 (+ 2301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:09:06.789533 (+ 43us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:09:06.790149 (+ 616us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:09:06.790633 (+ 484us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:09:06.791201 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:09:06.793262 (+ 2061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:09:06.794625 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:09:06.795631 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:09:06.795810 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:09:06.797209 (+ 1399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:09:06.797220 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:09:06.797239 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:09:06.798051 (+ 812us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:09:06.798182 (+ 131us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:09:06.798221 (+ 39us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:09:06.798338 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 09:09:06.798921 (+ 583us) server_negotiation.cc:299] Negotiation successful
0811 09:09:06.799264 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":618,"thread_start_us":347,"threads_started":1}
I20260811 09:09:06.810801 14388 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f22db52014084600a5e7fcc973232576: No bootstrap required, opened a new log
I20260811 09:09:06.821471 14388 raft_consensus.cc:348] T 00000000000000000000000000000000 P f22db52014084600a5e7fcc973232576 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f22db52014084600a5e7fcc973232576" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 46867 } }
I20260811 09:09:06.821902 14388 raft_consensus.cc:374] T 00000000000000000000000000000000 P f22db52014084600a5e7fcc973232576 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:09:06.822000 14388 raft_consensus.cc:729] T 00000000000000000000000000000000 P f22db52014084600a5e7fcc973232576 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f22db52014084600a5e7fcc973232576, State: Initialized, Role: FOLLOWER
I20260811 09:09:06.822669 14388 consensus_queue.cc:260] T 00000000000000000000000000000000 P f22db52014084600a5e7fcc973232576 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f22db52014084600a5e7fcc973232576" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 46867 } }
I20260811 09:09:06.822907 14388 raft_consensus.cc:388] T 00000000000000000000000000000000 P f22db52014084600a5e7fcc973232576 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:09:06.823055 14388 raft_consensus.cc:482] T 00000000000000000000000000000000 P f22db52014084600a5e7fcc973232576 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:09:06.823231 14388 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f22db52014084600a5e7fcc973232576 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:09:06.825956 14388 raft_consensus.cc:504] T 00000000000000000000000000000000 P f22db52014084600a5e7fcc973232576 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f22db52014084600a5e7fcc973232576" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 46867 } }
I20260811 09:09:06.826495 14388 leader_election.cc:302] T 00000000000000000000000000000000 P f22db52014084600a5e7fcc973232576 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f22db52014084600a5e7fcc973232576; no voters:
I20260811 09:09:06.827590 14388 leader_election.cc:288] T 00000000000000000000000000000000 P f22db52014084600a5e7fcc973232576 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:09:06.827771 14393 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f22db52014084600a5e7fcc973232576 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:09:06.828820 14393 raft_consensus.cc:686] T 00000000000000000000000000000000 P f22db52014084600a5e7fcc973232576 [term 1 LEADER]: Becoming Leader. State: Replica: f22db52014084600a5e7fcc973232576, State: Running, Role: LEADER
I20260811 09:09:06.829545 14393 consensus_queue.cc:237] T 00000000000000000000000000000000 P f22db52014084600a5e7fcc973232576 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f22db52014084600a5e7fcc973232576" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 46867 } }
I20260811 09:09:06.830152 14388 sys_catalog.cc:564] T 00000000000000000000000000000000 P f22db52014084600a5e7fcc973232576 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:09:06.833647 14398 sys_catalog.cc:455] T 00000000000000000000000000000000 P f22db52014084600a5e7fcc973232576 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f22db52014084600a5e7fcc973232576. Latest consensus state: current_term: 1 leader_uuid: "f22db52014084600a5e7fcc973232576" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f22db52014084600a5e7fcc973232576" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 46867 } } }
I20260811 09:09:06.834076 14398 sys_catalog.cc:458] T 00000000000000000000000000000000 P f22db52014084600a5e7fcc973232576 [sys.catalog]: This master's current role is: LEADER
I20260811 09:09:06.833851 14394 sys_catalog.cc:455] T 00000000000000000000000000000000 P f22db52014084600a5e7fcc973232576 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f22db52014084600a5e7fcc973232576" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f22db52014084600a5e7fcc973232576" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 46867 } } }
I20260811 09:09:06.834190 14394 sys_catalog.cc:458] T 00000000000000000000000000000000 P f22db52014084600a5e7fcc973232576 [sys.catalog]: This master's current role is: LEADER
I20260811 09:09:06.835036 14404 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:09:06.851449 14404 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:09:06.862892 14404 catalog_manager.cc:1347] Generated new cluster ID: c0df9525dc6044a3b022c8620b2a4113
I20260811 09:09:06.863026 14404 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:09:06.874796 14404 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:09:06.876681 14404 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:09:06.895275 14404 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f22db52014084600a5e7fcc973232576: Generated new TSK 0
I20260811 09:09:06.896456 14404 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 09:09:06.901144 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439250192320-29290-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439250192320-29290-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439250192320-29290-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439250192320-29290-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.154.129:0
--local_ip_for_outbound_sockets=127.28.154.129
--webserver_interface=127.28.154.129
--webserver_port=0
--tserver_master_addrs=127.28.154.190:46867
--builtin_ntp_servers=127.28.154.148:34091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260811 09:09:07.254658 14424 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:09:07.254964 14424 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:09:07.255074 14424 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:09:07.262606 14424 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:09:07.262719 14424 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:09:07.262868 14424 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.154.129
I20260811 09:09:07.272149 14424 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:34091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439250192320-29290-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439250192320-29290-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.154.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439250192320-29290-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.154.129
--webserver_port=0
--tserver_master_addrs=127.28.154.190:46867
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.154.129
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439250192320-29290-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:09:07.273497 14424 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:09:07.275298 14424 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:09:07.289515 14432 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:09:07.289569 14440 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:09:07.289436 14445 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:09:07.290690 14424 server_base.cc:1034] running on GCE node
I20260811 09:09:07.291369 14424 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:09:07.292472 14424 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:09:07.293778 14424 hybrid_clock.cc:648] HybridClock initialized: now 1786439347293728 us; error 46 us; skew 500 ppm
I20260811 09:09:07.294114 14424 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 09:09:07.296480 14424 webserver.cc:460] Webserver started at http://127.28.154.129:40487/ using document root <none> and password file <none>
I20260811 09:09:07.297176 14424 fs_manager.cc:359] Metadata directory not provided
I20260811 09:09:07.297262 14424 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:09:07.297549 14424 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:09:07.300371 14424 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439250192320-29290-0/minicluster-data/ts-0/data/instance:
uuid: "febb3eae22074f74924323b3df64beaf"
format_stamp: "Formatted at 2026-08-11 09:09:07 on dist-test-slave-xt4k"
server_key: "22fdc2405a6991052447d74b8ea8871d"
server_key_iv: "b63b2b7fdae64e1a6640efcb5a5b85c3"
server_key_version: "encryptionkey@0"
I20260811 09:09:07.301162 14424 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439250192320-29290-0/minicluster-data/ts-0/wal/instance:
uuid: "febb3eae22074f74924323b3df64beaf"
format_stamp: "Formatted at 2026-08-11 09:09:07 on dist-test-slave-xt4k"
server_key: "22fdc2405a6991052447d74b8ea8871d"
server_key_iv: "b63b2b7fdae64e1a6640efcb5a5b85c3"
server_key_version: "encryptionkey@0"
I20260811 09:09:07.306329 14424 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 09:09:07.309811 14453 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:09:07.311204 14424 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 09:09:07.311378 14424 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439250192320-29290-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439250192320-29290-0/minicluster-data/ts-0/wal
uuid: "febb3eae22074f74924323b3df64beaf"
format_stamp: "Formatted at 2026-08-11 09:09:07 on dist-test-slave-xt4k"
server_key: "22fdc2405a6991052447d74b8ea8871d"
server_key_iv: "b63b2b7fdae64e1a6640efcb5a5b85c3"
server_key_version: "encryptionkey@0"
I20260811 09:09:07.311625 14424 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439250192320-29290-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439250192320-29290-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439250192320-29290-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:09:07.328970 14424 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:09:07.329834 14424 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:09:07.330127 14424 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:09:07.331197 14424 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:09:07.332842 14424 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:09:07.332975 14424 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:09:07.333094 14424 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:09:07.333156 14424 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:09:07.364493 14424 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.129:33313
I20260811 09:09:07.364554 14650 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.129:33313 every 8 connection(s)
I20260811 09:09:07.366111 14424 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439250192320-29290-0/minicluster-data/ts-0/data/info.pb
I20260811 09:09:07.374334 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 14424
I20260811 09:09:07.374538 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439250192320-29290-0/minicluster-data/ts-0/wal/instance
I20260811 09:09:07.375157 29290 external_mini_cluster.cc:1428] Setting key 08d7e86a7043bb2f0e6dfd61a482ad37
I20260811 09:09:07.377842 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439250192320-29290-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439250192320-29290-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439250192320-29290-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439250192320-29290-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.154.130:0
--local_ip_for_outbound_sockets=127.28.154.130
--webserver_interface=127.28.154.130
--webserver_port=0
--tserver_master_addrs=127.28.154.190:46867
--builtin_ntp_servers=127.28.154.148:34091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260811 09:09:07.379760 14654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:09:07.369290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.129:42553 (local address 127.28.154.190:46867)
0811 09:09:07.369716 (+ 426us) server_negotiation.cc:206] Beginning negotiation
0811 09:09:07.369721 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:09:07.370919 (+ 1198us) server_negotiation.cc:407] Connection header received
0811 09:09:07.372760 (+ 1841us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:09:07.372766 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:09:07.372848 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:09:07.372947 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:09:07.374063 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:09:07.374860 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:09:07.376598 (+ 1738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:09:07.377090 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:09:07.377202 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:09:07.378757 (+ 1555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:09:07.378766 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:09:07.378770 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:09:07.378795 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:09:07.378844 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:09:07.378861 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:09:07.379069 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0811 09:09:07.379295 (+ 226us) server_negotiation.cc:299] Negotiation successful
0811 09:09:07.379404 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":313,"thread_start_us":211,"threads_started":1}
I20260811 09:09:07.380879 14657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:09:07.369846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:46867 (local address 127.28.154.129:42553)
0811 09:09:07.370749 (+ 903us) negotiation.cc:107] Waiting for socket to connect
0811 09:09:07.370793 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 09:09:07.372473 (+ 1680us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:09:07.373196 (+ 723us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:09:07.373205 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:09:07.373293 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:09:07.373883 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:09:07.373900 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:09:07.375006 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:09:07.375010 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:09:07.376448 (+ 1438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:09:07.376470 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:09:07.377312 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:09:07.377320 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:09:07.377561 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:09:07.378306 (+ 745us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 09:09:07.378309 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:09:07.378348 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 09:09:07.378444 (+ 96us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:09:07.379070 (+ 626us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:09:07.379083 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:09:07.379092 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 09:09:07.379266 (+ 174us) client_negotiation.cc:240] Negotiation successful
0811 09:09:07.379720 (+ 454us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":261,"threads_started":1}
I20260811 09:09:07.382476 14651 heartbeater.cc:347] Connected to a master server at 127.28.154.190:46867
I20260811 09:09:07.382766 14651 heartbeater.cc:464] Registering TS with master...
I20260811 09:09:07.383546 14651 heartbeater.cc:511] Master 127.28.154.190:46867 requested a full tablet report, sending...
I20260811 09:09:07.385708 14316 ts_manager.cc:194] Registered new tserver with Master: febb3eae22074f74924323b3df64beaf (127.28.154.129:33313)
I20260811 09:09:07.387255 14316 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.28.154.129:42553
W20260811 09:09:07.725214 14658 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:09:07.725525 14658 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:09:07.725664 14658 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:09:07.733290 14658 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:09:07.733430 14658 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:09:07.733580 14658 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.154.130
I20260811 09:09:07.742870 14658 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:34091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439250192320-29290-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439250192320-29290-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.154.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439250192320-29290-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.154.130
--webserver_port=0
--tserver_master_addrs=127.28.154.190:46867
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.154.130
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439250192320-29290-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:09:07.744238 14658 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:09:07.746052 14658 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:09:07.759261 14672 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:09:07.759361 14669 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:09:07.759333 14675 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:09:07.760303 14658 server_base.cc:1034] running on GCE node
I20260811 09:09:07.760984 14658 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:09:07.761963 14658 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:09:07.763190 14658 hybrid_clock.cc:648] HybridClock initialized: now 1786439347763127 us; error 52 us; skew 500 ppm
I20260811 09:09:07.763545 14658 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 09:09:07.765956 14658 webserver.cc:460] Webserver started at http://127.28.154.130:41267/ using document root <none> and password file <none>
I20260811 09:09:07.766673 14658 fs_manager.cc:359] Metadata directory not provided
I20260811 09:09:07.766780 14658 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:09:07.767119 14658 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:09:07.770177 14658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439250192320-29290-0/minicluster-data/ts-1/data/instance:
uuid: "3eca9516e3af43ef8e0e67ead4b5af00"
format_stamp: "Formatted at 2026-08-11 09:09:07 on dist-test-slave-xt4k"
server_key: "9d9aa42945d907dcf8dd99da3ec06550"
server_key_iv: "0e64f73b17c82bec4625f174c0a581ca"
server_key_version: "encryptionkey@0"
I20260811 09:09:07.771018 14658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439250192320-29290-0/minicluster-data/ts-1/wal/instance:
uuid: "3eca9516e3af43ef8e0e67ead4b5af00"
format_stamp: "Formatted at 2026-08-11 09:09:07 on dist-test-slave-xt4k"
server_key: "9d9aa42945d907dcf8dd99da3ec06550"
server_key_iv: "0e64f73b17c82bec4625f174c0a581ca"
server_key_version: "encryptionkey@0"
I20260811 09:09:07.776594 14658 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 09:09:07.780174 14689 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:09:07.781673 14658 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 09:09:07.781852 14658 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439250192320-29290-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439250192320-29290-0/minicluster-data/ts-1/wal
uuid: "3eca9516e3af43ef8e0e67ead4b5af00"
format_stamp: "Formatted at 2026-08-11 09:09:07 on dist-test-slave-xt4k"
server_key: "9d9aa42945d907dcf8dd99da3ec06550"
server_key_iv: "0e64f73b17c82bec4625f174c0a581ca"
server_key_version: "encryptionkey@0"
I20260811 09:09:07.782068 14658 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439250192320-29290-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439250192320-29290-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439250192320-29290-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:09:07.803910 14658 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:09:07.805042 14658 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:09:07.805370 14658 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:09:07.806397 14658 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:09:07.807910 14658 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:09:07.808007 14658 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:09:07.808121 14658 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:09:07.808190 14658 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:09:07.839591 14658 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.130:36491
I20260811 09:09:07.839672 14888 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.130:36491 every 8 connection(s)
I20260811 09:09:07.840936 14658 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439250192320-29290-0/minicluster-data/ts-1/data/info.pb
I20260811 09:09:07.847359 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 14658
I20260811 09:09:07.847549 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439250192320-29290-0/minicluster-data/ts-1/wal/instance
I20260811 09:09:07.848115 29290 external_mini_cluster.cc:1428] Setting key b7b08e036ff32df6d2f7b3f014ea4f7a
I20260811 09:09:07.851034 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439250192320-29290-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439250192320-29290-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439250192320-29290-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439250192320-29290-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.154.131:0
--local_ip_for_outbound_sockets=127.28.154.131
--webserver_interface=127.28.154.131
--webserver_port=0
--tserver_master_addrs=127.28.154.190:46867
--builtin_ntp_servers=127.28.154.148:34091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260811 09:09:07.853617 14654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:09:07.843462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.130:45089 (local address 127.28.154.190:46867)
0811 09:09:07.843638 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 09:09:07.843642 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:09:07.845086 (+ 1444us) server_negotiation.cc:407] Connection header received
0811 09:09:07.846843 (+ 1757us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:09:07.846849 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:09:07.846942 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:09:07.847074 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:09:07.848185 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:09:07.849121 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:09:07.850849 (+ 1728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:09:07.851308 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:09:07.851435 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:09:07.852811 (+ 1376us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:09:07.852815 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:09:07.852819 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:09:07.852837 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:09:07.852870 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:09:07.852882 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:09:07.852966 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 09:09:07.853197 (+ 231us) server_negotiation.cc:299] Negotiation successful
0811 09:09:07.853301 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 09:09:07.854656 14891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:09:07.843980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:46867 (local address 127.28.154.130:45089)
0811 09:09:07.844920 (+ 940us) negotiation.cc:107] Waiting for socket to connect
0811 09:09:07.844960 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 09:09:07.846552 (+ 1592us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:09:07.847255 (+ 703us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:09:07.847263 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:09:07.847352 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:09:07.847974 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:09:07.847992 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:09:07.849291 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:09:07.849302 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:09:07.850701 (+ 1399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:09:07.850715 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:09:07.851544 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:09:07.851552 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:09:07.851766 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:09:07.852492 (+ 726us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 09:09:07.852494 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:09:07.852537 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 09:09:07.852626 (+ 89us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:09:07.852976 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:09:07.852988 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:09:07.852997 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 09:09:07.853158 (+ 161us) client_negotiation.cc:240] Negotiation successful
0811 09:09:07.853575 (+ 417us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":502,"thread_start_us":309,"threads_started":1}
I20260811 09:09:07.856143 14889 heartbeater.cc:347] Connected to a master server at 127.28.154.190:46867
I20260811 09:09:07.856504 14889 heartbeater.cc:464] Registering TS with master...
I20260811 09:09:07.857378 14889 heartbeater.cc:511] Master 127.28.154.190:46867 requested a full tablet report, sending...
I20260811 09:09:07.858927 14316 ts_manager.cc:194] Registered new tserver with Master: 3eca9516e3af43ef8e0e67ead4b5af00 (127.28.154.130:36491)
I20260811 09:09:07.859948 14316 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.28.154.130:45089
W20260811 09:09:08.192901 14894 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:09:08.193219 14894 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:09:08.193339 14894 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:09:08.200623 14894 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:09:08.200738 14894 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:09:08.200883 14894 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.154.131
I20260811 09:09:08.210065 14894 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:34091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439250192320-29290-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439250192320-29290-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.154.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439250192320-29290-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.154.131
--webserver_port=0
--tserver_master_addrs=127.28.154.190:46867
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.154.131
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439250192320-29290-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:09:08.211351 14894 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:09:08.213114 14894 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:09:08.226110 14908 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:09:08.226145 14904 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:09:08.226166 14903 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:09:08.226743 14894 server_base.cc:1034] running on GCE node
I20260811 09:09:08.227335 14894 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:09:08.228302 14894 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:09:08.229607 14894 hybrid_clock.cc:648] HybridClock initialized: now 1786439348229537 us; error 36 us; skew 500 ppm
I20260811 09:09:08.229965 14894 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 09:09:08.232360 14894 webserver.cc:460] Webserver started at http://127.28.154.131:41181/ using document root <none> and password file <none>
I20260811 09:09:08.233162 14894 fs_manager.cc:359] Metadata directory not provided
I20260811 09:09:08.233258 14894 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:09:08.233600 14894 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:09:08.236629 14894 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439250192320-29290-0/minicluster-data/ts-2/data/instance:
uuid: "413ecf8093ed47dea7a774124a96b683"
format_stamp: "Formatted at 2026-08-11 09:09:08 on dist-test-slave-xt4k"
server_key: "88ced46df4cc087fedbca5e4aa80ef72"
server_key_iv: "54ac177389e92e916c3539a3a601f4e3"
server_key_version: "encryptionkey@0"
I20260811 09:09:08.237432 14894 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439250192320-29290-0/minicluster-data/ts-2/wal/instance:
uuid: "413ecf8093ed47dea7a774124a96b683"
format_stamp: "Formatted at 2026-08-11 09:09:08 on dist-test-slave-xt4k"
server_key: "88ced46df4cc087fedbca5e4aa80ef72"
server_key_iv: "54ac177389e92e916c3539a3a601f4e3"
server_key_version: "encryptionkey@0"
I20260811 09:09:08.242981 14894 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 09:09:08.246562 14917 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:09:08.247895 14894 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 09:09:08.248060 14894 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439250192320-29290-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439250192320-29290-0/minicluster-data/ts-2/wal
uuid: "413ecf8093ed47dea7a774124a96b683"
format_stamp: "Formatted at 2026-08-11 09:09:08 on dist-test-slave-xt4k"
server_key: "88ced46df4cc087fedbca5e4aa80ef72"
server_key_iv: "54ac177389e92e916c3539a3a601f4e3"
server_key_version: "encryptionkey@0"
I20260811 09:09:08.248266 14894 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439250192320-29290-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439250192320-29290-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439250192320-29290-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:09:08.264068 14894 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:09:08.265038 14894 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:09:08.265324 14894 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:09:08.266301 14894 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:09:08.267787 14894 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:09:08.267885 14894 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:09:08.268004 14894 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:09:08.268074 14894 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:09:08.301232 14894 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.131:40655
I20260811 09:09:08.301311 15104 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.131:40655 every 8 connection(s)
I20260811 09:09:08.302533 14894 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439250192320-29290-0/minicluster-data/ts-2/data/info.pb
I20260811 09:09:08.310230 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 14894
I20260811 09:09:08.310412 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786439250192320-29290-0/minicluster-data/ts-2/wal/instance
I20260811 09:09:08.310843 29290 external_mini_cluster.cc:1428] Setting key a2e4fe47dee62255c7968fce80aac558
I20260811 09:09:08.315366 14654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:09:08.305172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.131:33561 (local address 127.28.154.190:46867)
0811 09:09:08.305379 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0811 09:09:08.305384 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:09:08.306776 (+ 1392us) server_negotiation.cc:407] Connection header received
0811 09:09:08.308938 (+ 2162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:09:08.308944 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:09:08.309028 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:09:08.309122 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:09:08.310242 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:09:08.311077 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:09:08.312683 (+ 1606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:09:08.313113 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:09:08.313228 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:09:08.314570 (+ 1342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:09:08.314576 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:09:08.314579 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:09:08.314599 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:09:08.314632 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:09:08.314645 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:09:08.314721 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 09:09:08.314936 (+ 215us) server_negotiation.cc:299] Negotiation successful
0811 09:09:08.315039 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 09:09:08.316527 15109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:09:08.305679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:46867 (local address 127.28.154.131:33561)
0811 09:09:08.306605 (+ 926us) negotiation.cc:107] Waiting for socket to connect
0811 09:09:08.306642 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 09:09:08.308660 (+ 2018us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:09:08.309384 (+ 724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:09:08.309393 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:09:08.309480 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:09:08.310066 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:09:08.310083 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:09:08.311236 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:09:08.311242 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:09:08.312540 (+ 1298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:09:08.312554 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:09:08.313329 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:09:08.313336 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:09:08.313583 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:09:08.314245 (+ 662us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 09:09:08.314248 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:09:08.314289 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 09:09:08.314368 (+ 79us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:09:08.314734 (+ 366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:09:08.314745 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:09:08.314753 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 09:09:08.314909 (+ 156us) client_negotiation.cc:240] Negotiation successful
0811 09:09:08.315320 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":295,"threads_started":1}
I20260811 09:09:08.317899 15105 heartbeater.cc:347] Connected to a master server at 127.28.154.190:46867
I20260811 09:09:08.318229 15105 heartbeater.cc:464] Registering TS with master...
I20260811 09:09:08.319092 15105 heartbeater.cc:511] Master 127.28.154.190:46867 requested a full tablet report, sending...
I20260811 09:09:08.320595 14316 ts_manager.cc:194] Registered new tserver with Master: 413ecf8093ed47dea7a774124a96b683 (127.28.154.131:40655)
I20260811 09:09:08.321405 14316 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.28.154.131:33561
I20260811 09:09:08.326347 29290 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 09:09:08.333021 14654 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:09:08.331113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33962 (local address 127.28.154.190:46867)
0811 09:09:08.331333 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0811 09:09:08.331337 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:09:08.331835 (+ 498us) server_negotiation.cc:407] Connection header received
0811 09:09:08.331977 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:09:08.331981 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:09:08.332065 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:09:08.332151 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:09:08.332712 (+ 561us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33962: BlockingRecv error: recv got EOF from 127.0.0.1:33962 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
I20260811 09:09:08.333673 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 14424
I20260811 09:09:08.341917 15125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:09:08.334699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58052 (local address 127.28.154.129:33313)
0811 09:09:08.335243 (+ 544us) server_negotiation.cc:206] Beginning negotiation
0811 09:09:08.335251 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:09:08.335278 (+ 27us) server_negotiation.cc:407] Connection header received
0811 09:09:08.335341 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:09:08.335348 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:09:08.335562 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:09:08.335695 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:09:08.336071 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:09:08.337256 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:09:08.338492 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:09:08.339112 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:09:08.339216 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:09:08.340333 (+ 1117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:09:08.340385 (+ 52us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:09:08.340408 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:09:08.341059 (+ 651us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:09:08.341115 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:09:08.341137 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:09:08.341233 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 09:09:08.341471 (+ 238us) server_negotiation.cc:299] Negotiation successful
0811 09:09:08.341614 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":404,"thread_start_us":281,"threads_started":1}
I20260811 09:09:08.343379 14643 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:09:08.424177 14651 heartbeater.cc:503] Master 127.28.154.190:46867 was elected leader, sending a full tablet report...
I20260811 09:09:08.425334 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 14424
W20260811 09:09:08.443848 14287 connection.cc:535] server connection from 127.28.154.129:42553 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 09:09:08.444167 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 14658
I20260811 09:09:08.451874 15129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:09:08.445085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55602 (local address 127.28.154.130:36491)
0811 09:09:08.445573 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0811 09:09:08.445580 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:09:08.445606 (+ 26us) server_negotiation.cc:407] Connection header received
0811 09:09:08.445680 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:09:08.445690 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:09:08.445904 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:09:08.446056 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:09:08.446457 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:09:08.447446 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:09:08.448700 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:09:08.449278 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:09:08.449397 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:09:08.450393 (+ 996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:09:08.450403 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:09:08.450418 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:09:08.451015 (+ 597us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:09:08.451073 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:09:08.451094 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:09:08.451194 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 09:09:08.451348 (+ 154us) server_negotiation.cc:299] Negotiation successful
0811 09:09:08.451506 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":238,"threads_started":1}
I20260811 09:09:08.453223 14881 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:09:08.535356 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 14658
I20260811 09:09:08.561765 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 14894
I20260811 09:09:08.570405 15132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:09:08.562822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53880 (local address 127.28.154.131:40655)
0811 09:09:08.563517 (+ 695us) server_negotiation.cc:206] Beginning negotiation
0811 09:09:08.563524 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:09:08.563556 (+ 32us) server_negotiation.cc:407] Connection header received
0811 09:09:08.563628 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:09:08.563635 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:09:08.563888 (+ 253us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:09:08.564059 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:09:08.564550 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:09:08.565625 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:09:08.566888 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:09:08.567490 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:09:08.567617 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:09:08.568853 (+ 1236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:09:08.568863 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:09:08.568878 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:09:08.569506 (+ 628us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:09:08.569568 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:09:08.569590 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:09:08.569699 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 09:09:08.569882 (+ 183us) server_negotiation.cc:299] Negotiation successful
0811 09:09:08.570053 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":536,"thread_start_us":368,"threads_started":1}
I20260811 09:09:08.571782 15098 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:09:08.654909 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 14894
I20260811 09:09:08.680449 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 14258
I20260811 09:09:08.681787 14377 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:09:08.785315 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 14258
2026-08-11T09:09:08Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2570 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-11T09:09:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:09:08Z Disabled control of system clock
I20260811 09:09:08.876469 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439250192320-29290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439250192320-29290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439250192320-29290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.154.190:33015
--webserver_interface=127.28.154.190
--webserver_port=0
--builtin_ntp_servers=127.28.154.148:39679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.154.190:33015
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260811 09:09:09.226745 15146 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:09:09.227046 15146 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:09:09.227171 15146 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:09:09.234596 15146 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:09:09.234715 15146 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:09:09.234762 15146 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:09:09.234846 15146 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:09:09.234906 15146 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:09:09.244441 15146 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:39679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439250192320-29290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439250192320-29290-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.154.190:33015
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.154.190:33015
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.154.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439250192320-29290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:09:09.245843 15146 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:09:09.247723 15146 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:09:09.259945 15164 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:09:09.259974 15167 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:09:09.260062 15161 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:09:09.260689 15146 server_base.cc:1034] running on GCE node
I20260811 09:09:09.261526 15146 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:09:09.263109 15146 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:09:09.264369 15146 hybrid_clock.cc:648] HybridClock initialized: now 1786439349264296 us; error 44 us; skew 500 ppm
I20260811 09:09:09.264768 15146 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 09:09:09.267129 15146 webserver.cc:460] Webserver started at http://127.28.154.190:33883/ using document root <none> and password file <none>
I20260811 09:09:09.267856 15146 fs_manager.cc:359] Metadata directory not provided
I20260811 09:09:09.267958 15146 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:09:09.268285 15146 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:09:09.271373 15146 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439250192320-29290-0/minicluster-data/master-0/data/instance:
uuid: "ab7da8bdce0743eda19a2637ee0cf0e2"
format_stamp: "Formatted at 2026-08-11 09:09:09 on dist-test-slave-xt4k"
server_key: "ef143af4109f10392b1d53501e3db85d"
server_key_iv: "697144c3a3483436b643c51a40f2b520"
server_key_version: "encryptionkey@0"
I20260811 09:09:09.272205 15146 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439250192320-29290-0/minicluster-data/master-0/wal/instance:
uuid: "ab7da8bdce0743eda19a2637ee0cf0e2"
format_stamp: "Formatted at 2026-08-11 09:09:09 on dist-test-slave-xt4k"
server_key: "ef143af4109f10392b1d53501e3db85d"
server_key_iv: "697144c3a3483436b643c51a40f2b520"
server_key_version: "encryptionkey@0"
I20260811 09:09:09.277745 15146 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260811 09:09:09.281533 15176 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:09:09.283103 15146 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 09:09:09.283290 15146 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439250192320-29290-0/minicluster-data/master-0/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439250192320-29290-0/minicluster-data/master-0/wal
uuid: "ab7da8bdce0743eda19a2637ee0cf0e2"
format_stamp: "Formatted at 2026-08-11 09:09:09 on dist-test-slave-xt4k"
server_key: "ef143af4109f10392b1d53501e3db85d"
server_key_iv: "697144c3a3483436b643c51a40f2b520"
server_key_version: "encryptionkey@0"
I20260811 09:09:09.283536 15146 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439250192320-29290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439250192320-29290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439250192320-29290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:09:09.290211 15146 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:09:09.291077 15146 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:09:09.291371 15146 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:09:09.309226 15146 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.190:33015
I20260811 09:09:09.309243 15277 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.190:33015 every 8 connection(s)
I20260811 09:09:09.310616 15146 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
I20260811 09:09:09.314589 15278 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:09:09.315112 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 15146
I20260811 09:09:09.315311 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439250192320-29290-0/minicluster-data/master-0/wal/instance
I20260811 09:09:09.316184 29290 external_mini_cluster.cc:1428] Setting key c53e10de3ab53a130137797a34179277
I20260811 09:09:09.325074 15283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:09:09.318472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40272 (local address 127.28.154.190:33015)
0811 09:09:09.319496 (+ 1024us) server_negotiation.cc:206] Beginning negotiation
0811 09:09:09.319508 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 09:09:09.319552 (+ 44us) server_negotiation.cc:407] Connection header received
0811 09:09:09.321385 (+ 1833us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:09:09.321420 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:09:09.321884 (+ 464us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:09:09.322278 (+ 394us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:09:09.323343 (+ 1065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:09:09.323350 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:09:09.323364 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:09:09.323385 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:09:09.323457 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:09:09.323476 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:09:09.323690 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0811 09:09:09.323821 (+ 131us) server_negotiation.cc:299] Negotiation successful
0811 09:09:09.324079 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":564,"thread_start_us":345,"threads_started":1}
I20260811 09:09:09.328683 15278 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab7da8bdce0743eda19a2637ee0cf0e2: Bootstrap starting.
I20260811 09:09:09.331640 15278 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ab7da8bdce0743eda19a2637ee0cf0e2: Neither blocks nor log segments found. Creating new log.
I20260811 09:09:09.332681 15278 log.cc:824] T 00000000000000000000000000000000 P ab7da8bdce0743eda19a2637ee0cf0e2: Log is configured to *not* fsync() on all Append() calls
I20260811 09:09:09.335623 15278 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ab7da8bdce0743eda19a2637ee0cf0e2: No bootstrap required, opened a new log
I20260811 09:09:09.342931 15278 raft_consensus.cc:348] T 00000000000000000000000000000000 P ab7da8bdce0743eda19a2637ee0cf0e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab7da8bdce0743eda19a2637ee0cf0e2" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 33015 } }
I20260811 09:09:09.343310 15278 raft_consensus.cc:374] T 00000000000000000000000000000000 P ab7da8bdce0743eda19a2637ee0cf0e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:09:09.343413 15278 raft_consensus.cc:729] T 00000000000000000000000000000000 P ab7da8bdce0743eda19a2637ee0cf0e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab7da8bdce0743eda19a2637ee0cf0e2, State: Initialized, Role: FOLLOWER
I20260811 09:09:09.344089 15278 consensus_queue.cc:260] T 00000000000000000000000000000000 P ab7da8bdce0743eda19a2637ee0cf0e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab7da8bdce0743eda19a2637ee0cf0e2" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 33015 } }
I20260811 09:09:09.344321 15278 raft_consensus.cc:388] T 00000000000000000000000000000000 P ab7da8bdce0743eda19a2637ee0cf0e2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:09:09.344463 15278 raft_consensus.cc:482] T 00000000000000000000000000000000 P ab7da8bdce0743eda19a2637ee0cf0e2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:09:09.344596 15278 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ab7da8bdce0743eda19a2637ee0cf0e2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:09:09.346673 15278 raft_consensus.cc:504] T 00000000000000000000000000000000 P ab7da8bdce0743eda19a2637ee0cf0e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab7da8bdce0743eda19a2637ee0cf0e2" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 33015 } }
I20260811 09:09:09.347127 15278 leader_election.cc:302] T 00000000000000000000000000000000 P ab7da8bdce0743eda19a2637ee0cf0e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ab7da8bdce0743eda19a2637ee0cf0e2; no voters:
I20260811 09:09:09.347730 15278 leader_election.cc:288] T 00000000000000000000000000000000 P ab7da8bdce0743eda19a2637ee0cf0e2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:09:09.347875 15290 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ab7da8bdce0743eda19a2637ee0cf0e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:09:09.349009 15290 raft_consensus.cc:686] T 00000000000000000000000000000000 P ab7da8bdce0743eda19a2637ee0cf0e2 [term 1 LEADER]: Becoming Leader. State: Replica: ab7da8bdce0743eda19a2637ee0cf0e2, State: Running, Role: LEADER
I20260811 09:09:09.349572 15290 consensus_queue.cc:237] T 00000000000000000000000000000000 P ab7da8bdce0743eda19a2637ee0cf0e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab7da8bdce0743eda19a2637ee0cf0e2" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 33015 } }
I20260811 09:09:09.349995 15278 sys_catalog.cc:564] T 00000000000000000000000000000000 P ab7da8bdce0743eda19a2637ee0cf0e2 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:09:09.352223 15296 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab7da8bdce0743eda19a2637ee0cf0e2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ab7da8bdce0743eda19a2637ee0cf0e2. Latest consensus state: current_term: 1 leader_uuid: "ab7da8bdce0743eda19a2637ee0cf0e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab7da8bdce0743eda19a2637ee0cf0e2" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 33015 } } }
I20260811 09:09:09.352617 15296 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab7da8bdce0743eda19a2637ee0cf0e2 [sys.catalog]: This master's current role is: LEADER
I20260811 09:09:09.352412 15292 sys_catalog.cc:455] T 00000000000000000000000000000000 P ab7da8bdce0743eda19a2637ee0cf0e2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ab7da8bdce0743eda19a2637ee0cf0e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ab7da8bdce0743eda19a2637ee0cf0e2" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 33015 } } }
I20260811 09:09:09.352768 15292 sys_catalog.cc:458] T 00000000000000000000000000000000 P ab7da8bdce0743eda19a2637ee0cf0e2 [sys.catalog]: This master's current role is: LEADER
I20260811 09:09:09.353760 15309 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:09:09.359359 15309 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:09:09.369728 15309 catalog_manager.cc:1347] Generated new cluster ID: 9c3911d14560405d99a6f87876d5ccb8
I20260811 09:09:09.369956 15309 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:09:09.382877 15309 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:09:09.383306 15309 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:09:09.390523 15309 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ab7da8bdce0743eda19a2637ee0cf0e2: Generated new TSK 0
I20260811 09:09:09.391579 15309 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 09:09:09.405505 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439250192320-29290-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439250192320-29290-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439250192320-29290-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439250192320-29290-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.154.129:0
--local_ip_for_outbound_sockets=127.28.154.129
--webserver_interface=127.28.154.129
--webserver_port=0
--tserver_master_addrs=127.28.154.190:33015
--builtin_ntp_servers=127.28.154.148:39679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260811 09:09:09.743152 15335 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:09:09.743462 15335 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:09:09.743592 15335 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:09:09.751539 15335 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:09:09.751669 15335 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:09:09.751809 15335 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.154.129
I20260811 09:09:09.761410 15335 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:39679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439250192320-29290-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439250192320-29290-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.154.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439250192320-29290-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.154.129
--webserver_port=0
--tserver_master_addrs=127.28.154.190:33015
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.154.129
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439250192320-29290-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:09:09.762836 15335 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:09:09.764725 15335 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:09:09.778328 15342 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:09:09.778236 15343 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:09:09.778270 15346 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:09:09.779326 15335 server_base.cc:1034] running on GCE node
I20260811 09:09:09.779970 15335 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:09:09.781188 15335 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:09:09.782449 15335 hybrid_clock.cc:648] HybridClock initialized: now 1786439349782382 us; error 48 us; skew 500 ppm
I20260811 09:09:09.782825 15335 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 09:09:09.785270 15335 webserver.cc:460] Webserver started at http://127.28.154.129:40927/ using document root <none> and password file <none>
I20260811 09:09:09.785980 15335 fs_manager.cc:359] Metadata directory not provided
I20260811 09:09:09.786073 15335 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:09:09.786357 15335 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:09:09.791594 15335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439250192320-29290-0/minicluster-data/ts-0/data/instance:
uuid: "529ed8797a584d8fa6215dab472c7cca"
format_stamp: "Formatted at 2026-08-11 09:09:09 on dist-test-slave-xt4k"
server_key: "91e8bf6286610b955ad70aa42a345786"
server_key_iv: "2b2fbd3204504a24a72a589ad708b925"
server_key_version: "encryptionkey@0"
I20260811 09:09:09.792688 15335 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439250192320-29290-0/minicluster-data/ts-0/wal/instance:
uuid: "529ed8797a584d8fa6215dab472c7cca"
format_stamp: "Formatted at 2026-08-11 09:09:09 on dist-test-slave-xt4k"
server_key: "91e8bf6286610b955ad70aa42a345786"
server_key_iv: "2b2fbd3204504a24a72a589ad708b925"
server_key_version: "encryptionkey@0"
I20260811 09:09:09.798533 15335 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 09:09:09.802202 15356 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:09:09.803745 15335 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 09:09:09.803913 15335 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439250192320-29290-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439250192320-29290-0/minicluster-data/ts-0/wal
uuid: "529ed8797a584d8fa6215dab472c7cca"
format_stamp: "Formatted at 2026-08-11 09:09:09 on dist-test-slave-xt4k"
server_key: "91e8bf6286610b955ad70aa42a345786"
server_key_iv: "2b2fbd3204504a24a72a589ad708b925"
server_key_version: "encryptionkey@0"
I20260811 09:09:09.804143 15335 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439250192320-29290-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439250192320-29290-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439250192320-29290-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:09:09.809970 15335 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:09:09.810833 15335 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:09:09.811100 15335 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:09:09.812098 15335 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:09:09.813635 15335 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:09:09.813735 15335 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:09:09.813848 15335 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:09:09.813920 15335 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:09:09.844084 15335 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.129:34721
I20260811 09:09:09.844154 15540 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.129:34721 every 8 connection(s)
I20260811 09:09:09.845438 15335 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439250192320-29290-0/minicluster-data/ts-0/data/info.pb
I20260811 09:09:09.853932 15547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:09:09.848037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.129:47573 (local address 127.28.154.190:33015)
0811 09:09:09.848515 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0811 09:09:09.848520 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:09:09.849723 (+ 1203us) server_negotiation.cc:407] Connection header received
0811 09:09:09.851546 (+ 1823us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:09:09.851551 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:09:09.851625 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:09:09.851727 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:09:09.853078 (+ 1351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:09:09.853085 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:09:09.853089 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:09:09.853113 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:09:09.853180 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:09:09.853202 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:09:09.853329 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0811 09:09:09.853509 (+ 180us) server_negotiation.cc:299] Negotiation successful
0811 09:09:09.853574 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":216,"threads_started":1}
I20260811 09:09:09.854363 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 15335
I20260811 09:09:09.854579 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439250192320-29290-0/minicluster-data/ts-0/wal/instance
I20260811 09:09:09.855110 15550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:09:09.848623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:33015 (local address 127.28.154.129:47573)
0811 09:09:09.849571 (+ 948us) negotiation.cc:107] Waiting for socket to connect
0811 09:09:09.849609 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 09:09:09.851205 (+ 1596us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:09:09.851954 (+ 749us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:09:09.851964 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:09:09.852063 (+ 99us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:09:09.852727 (+ 664us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 09:09:09.852730 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:09:09.852769 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 09:09:09.852861 (+ 92us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:09:09.853296 (+ 435us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:09:09.853303 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:09:09.853310 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 09:09:09.853574 (+ 264us) client_negotiation.cc:240] Negotiation successful
0811 09:09:09.853831 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":284,"threads_started":1}
I20260811 09:09:09.855378 29290 external_mini_cluster.cc:1428] Setting key bbc29548ac4b21bf70fd208e001e7dac
I20260811 09:09:09.856877 15541 heartbeater.cc:347] Connected to a master server at 127.28.154.190:33015
I20260811 09:09:09.857206 15541 heartbeater.cc:464] Registering TS with master...
I20260811 09:09:09.857796 15541 heartbeater.cc:511] Master 127.28.154.190:33015 requested a full tablet report, sending...
I20260811 09:09:09.858174 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439250192320-29290-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439250192320-29290-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439250192320-29290-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439250192320-29290-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.154.130:0
--local_ip_for_outbound_sockets=127.28.154.130
--webserver_interface=127.28.154.130
--webserver_port=0
--tserver_master_addrs=127.28.154.190:33015
--builtin_ntp_servers=127.28.154.148:39679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260811 09:09:09.860025 15214 ts_manager.cc:194] Registered new tserver with Master: 529ed8797a584d8fa6215dab472c7cca (127.28.154.129:34721)
W20260811 09:09:10.201478 15553 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:09:10.201776 15553 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:09:10.201864 15553 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:09:10.209314 15553 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:09:10.209434 15553 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:09:10.209547 15553 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.154.130
I20260811 09:09:10.218799 15553 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:39679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439250192320-29290-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439250192320-29290-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.154.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439250192320-29290-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.154.130
--webserver_port=0
--tserver_master_addrs=127.28.154.190:33015
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.154.130
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439250192320-29290-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:09:10.220126 15553 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:09:10.221866 15553 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:09:10.234973 15566 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:09:10.235044 15564 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:09:10.235211 15561 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:09:10.236217 15553 server_base.cc:1034] running on GCE node
I20260811 09:09:10.236922 15553 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:09:10.237946 15553 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:09:10.239174 15553 hybrid_clock.cc:648] HybridClock initialized: now 1786439350239113 us; error 47 us; skew 500 ppm
I20260811 09:09:10.239496 15553 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 09:09:10.241963 15553 webserver.cc:460] Webserver started at http://127.28.154.130:40475/ using document root <none> and password file <none>
I20260811 09:09:10.242713 15553 fs_manager.cc:359] Metadata directory not provided
I20260811 09:09:10.242825 15553 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:09:10.243166 15553 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:09:10.246253 15553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439250192320-29290-0/minicluster-data/ts-1/data/instance:
uuid: "ce31c66c62c34e8cbf32fb74daf7c23e"
format_stamp: "Formatted at 2026-08-11 09:09:10 on dist-test-slave-xt4k"
server_key: "185369fcd44be438551c1edce727c283"
server_key_iv: "907e028de3a6347db57524bbdeb36c77"
server_key_version: "encryptionkey@0"
I20260811 09:09:10.247059 15553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439250192320-29290-0/minicluster-data/ts-1/wal/instance:
uuid: "ce31c66c62c34e8cbf32fb74daf7c23e"
format_stamp: "Formatted at 2026-08-11 09:09:10 on dist-test-slave-xt4k"
server_key: "185369fcd44be438551c1edce727c283"
server_key_iv: "907e028de3a6347db57524bbdeb36c77"
server_key_version: "encryptionkey@0"
I20260811 09:09:10.252386 15553 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 09:09:10.255970 15577 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:09:10.257365 15553 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 09:09:10.257517 15553 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439250192320-29290-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439250192320-29290-0/minicluster-data/ts-1/wal
uuid: "ce31c66c62c34e8cbf32fb74daf7c23e"
format_stamp: "Formatted at 2026-08-11 09:09:10 on dist-test-slave-xt4k"
server_key: "185369fcd44be438551c1edce727c283"
server_key_iv: "907e028de3a6347db57524bbdeb36c77"
server_key_version: "encryptionkey@0"
I20260811 09:09:10.257732 15553 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439250192320-29290-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439250192320-29290-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439250192320-29290-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:09:10.263276 15553 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:09:10.264089 15553 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:09:10.264377 15553 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:09:10.265280 15553 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:09:10.266673 15553 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:09:10.266767 15553 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:09:10.266870 15553 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:09:10.266937 15553 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:09:10.299419 15553 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.130:45277
I20260811 09:09:10.299470 15789 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.130:45277 every 8 connection(s)
I20260811 09:09:10.300774 15553 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439250192320-29290-0/minicluster-data/ts-1/data/info.pb
I20260811 09:09:10.307289 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 15553
I20260811 09:09:10.307440 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439250192320-29290-0/minicluster-data/ts-1/wal/instance
I20260811 09:09:10.307924 29290 external_mini_cluster.cc:1428] Setting key 327943d6fe61ce127f3634f6cd0de8a9
I20260811 09:09:10.309619 15547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:09:10.303426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.130:58963 (local address 127.28.154.190:33015)
0811 09:09:10.303670 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0811 09:09:10.303676 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:09:10.305116 (+ 1440us) server_negotiation.cc:407] Connection header received
0811 09:09:10.306842 (+ 1726us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:09:10.306848 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:09:10.306949 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:09:10.307077 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:09:10.308646 (+ 1569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:09:10.308653 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:09:10.308658 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:09:10.308697 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:09:10.308744 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:09:10.308762 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:09:10.308895 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0811 09:09:10.309206 (+ 311us) server_negotiation.cc:299] Negotiation successful
0811 09:09:10.309275 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260811 09:09:10.310633 15798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:09:10.304000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:33015 (local address 127.28.154.130:58963)
0811 09:09:10.304953 (+ 953us) negotiation.cc:107] Waiting for socket to connect
0811 09:09:10.304990 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 09:09:10.306518 (+ 1528us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:09:10.307380 (+ 862us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:09:10.307391 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:09:10.307515 (+ 124us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:09:10.308218 (+ 703us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 09:09:10.308221 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:09:10.308260 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 09:09:10.308395 (+ 135us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:09:10.308882 (+ 487us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:09:10.308895 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:09:10.308903 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 09:09:10.309271 (+ 368us) client_negotiation.cc:240] Negotiation successful
0811 09:09:10.309535 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":510,"thread_start_us":323,"threads_started":1}
I20260811 09:09:10.311244 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439250192320-29290-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439250192320-29290-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439250192320-29290-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439250192320-29290-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.154.131:0
--local_ip_for_outbound_sockets=127.28.154.131
--webserver_interface=127.28.154.131
--webserver_port=0
--tserver_master_addrs=127.28.154.190:33015
--builtin_ntp_servers=127.28.154.148:39679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260811 09:09:10.311985 15792 heartbeater.cc:347] Connected to a master server at 127.28.154.190:33015
I20260811 09:09:10.312270 15792 heartbeater.cc:464] Registering TS with master...
I20260811 09:09:10.312889 15792 heartbeater.cc:511] Master 127.28.154.190:33015 requested a full tablet report, sending...
I20260811 09:09:10.314283 15214 ts_manager.cc:194] Registered new tserver with Master: ce31c66c62c34e8cbf32fb74daf7c23e (127.28.154.130:45277)
W20260811 09:09:10.654382 15799 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:09:10.654695 15799 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:09:10.654812 15799 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:09:10.662402 15799 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:09:10.662508 15799 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:09:10.662643 15799 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.154.131
I20260811 09:09:10.672212 15799 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:39679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439250192320-29290-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439250192320-29290-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.154.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439250192320-29290-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.154.131
--webserver_port=0
--tserver_master_addrs=127.28.154.190:33015
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.154.131
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439250192320-29290-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:09:10.673686 15799 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:09:10.675552 15799 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:09:10.689527 15816 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:09:10.689666 15815 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:09:10.690127 15818 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:09:10.690332 15799 server_base.cc:1034] running on GCE node
I20260811 09:09:10.691181 15799 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:09:10.692183 15799 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:09:10.693404 15799 hybrid_clock.cc:648] HybridClock initialized: now 1786439350693347 us; error 57 us; skew 500 ppm
I20260811 09:09:10.693783 15799 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 09:09:10.696110 15799 webserver.cc:460] Webserver started at http://127.28.154.131:45107/ using document root <none> and password file <none>
I20260811 09:09:10.696902 15799 fs_manager.cc:359] Metadata directory not provided
I20260811 09:09:10.697007 15799 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:09:10.697320 15799 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:09:10.700618 15799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439250192320-29290-0/minicluster-data/ts-2/data/instance:
uuid: "2f61b0598f974ff6873864a5fd77b250"
format_stamp: "Formatted at 2026-08-11 09:09:10 on dist-test-slave-xt4k"
server_key: "d1b41d064cc9b4c3f3b320b7a0ced35c"
server_key_iv: "f0ed4634243517bba859f28e7c3aa87f"
server_key_version: "encryptionkey@0"
I20260811 09:09:10.701459 15799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439250192320-29290-0/minicluster-data/ts-2/wal/instance:
uuid: "2f61b0598f974ff6873864a5fd77b250"
format_stamp: "Formatted at 2026-08-11 09:09:10 on dist-test-slave-xt4k"
server_key: "d1b41d064cc9b4c3f3b320b7a0ced35c"
server_key_iv: "f0ed4634243517bba859f28e7c3aa87f"
server_key_version: "encryptionkey@0"
I20260811 09:09:10.707122 15799 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 09:09:10.710978 15831 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:09:10.712422 15799 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 09:09:10.712608 15799 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439250192320-29290-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439250192320-29290-0/minicluster-data/ts-2/wal
uuid: "2f61b0598f974ff6873864a5fd77b250"
format_stamp: "Formatted at 2026-08-11 09:09:10 on dist-test-slave-xt4k"
server_key: "d1b41d064cc9b4c3f3b320b7a0ced35c"
server_key_iv: "f0ed4634243517bba859f28e7c3aa87f"
server_key_version: "encryptionkey@0"
I20260811 09:09:10.712829 15799 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439250192320-29290-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439250192320-29290-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439250192320-29290-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:09:10.718748 15799 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:09:10.719579 15799 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:09:10.719842 15799 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:09:10.720918 15799 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:09:10.722393 15799 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:09:10.722491 15799 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:09:10.722597 15799 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:09:10.722668 15799 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:09:10.753917 15799 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.131:33073
I20260811 09:09:10.753965 16045 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.131:33073 every 8 connection(s)
I20260811 09:09:10.755414 15799 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439250192320-29290-0/minicluster-data/ts-2/data/info.pb
I20260811 09:09:10.761567 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 15799
I20260811 09:09:10.761735 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786439250192320-29290-0/minicluster-data/ts-2/wal/instance
I20260811 09:09:10.762250 29290 external_mini_cluster.cc:1428] Setting key fb9e372c66e39ee9d9990a9d8ae4f976
I20260811 09:09:10.763711 15547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:09:10.757984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.131:48089 (local address 127.28.154.190:33015)
0811 09:09:10.758200 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0811 09:09:10.758206 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:09:10.759521 (+ 1315us) server_negotiation.cc:407] Connection header received
0811 09:09:10.761310 (+ 1789us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:09:10.761316 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:09:10.761391 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:09:10.761521 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:09:10.762872 (+ 1351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:09:10.762879 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:09:10.762883 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:09:10.762904 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:09:10.762949 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:09:10.762967 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:09:10.763075 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 09:09:10.763343 (+ 268us) server_negotiation.cc:299] Negotiation successful
0811 09:09:10.763410 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 09:09:10.764822 16052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:09:10.758445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:33015 (local address 127.28.154.131:48089)
0811 09:09:10.759366 (+ 921us) negotiation.cc:107] Waiting for socket to connect
0811 09:09:10.759410 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 09:09:10.761015 (+ 1605us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:09:10.761743 (+ 728us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:09:10.761751 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:09:10.761856 (+ 105us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:09:10.762568 (+ 712us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 09:09:10.762570 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:09:10.762610 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 09:09:10.762700 (+ 90us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:09:10.763090 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:09:10.763101 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:09:10.763109 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 09:09:10.763368 (+ 259us) client_negotiation.cc:240] Negotiation successful
0811 09:09:10.763628 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":273,"threads_started":1}
I20260811 09:09:10.766145 16047 heartbeater.cc:347] Connected to a master server at 127.28.154.190:33015
I20260811 09:09:10.766501 16047 heartbeater.cc:464] Registering TS with master...
I20260811 09:09:10.767132 16047 heartbeater.cc:511] Master 127.28.154.190:33015 requested a full tablet report, sending...
I20260811 09:09:10.768548 15214 ts_manager.cc:194] Registered new tserver with Master: 2f61b0598f974ff6873864a5fd77b250 (127.28.154.131:33073)
I20260811 09:09:10.777700 29290 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 09:09:10.783648 15547 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:09:10.782211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40282 (local address 127.28.154.190:33015)
0811 09:09:10.782436 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0811 09:09:10.782443 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:09:10.782846 (+ 403us) server_negotiation.cc:407] Connection header received
0811 09:09:10.782971 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:09:10.782974 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:09:10.783042 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:09:10.783120 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:09:10.783392 (+ 272us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40282: BlockingRecv error: recv got EOF from 127.0.0.1:40282 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 09:09:10.784578 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 15335
I20260811 09:09:10.788132 16067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:09:10.785616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52562 (local address 127.28.154.129:34721)
0811 09:09:10.786145 (+ 529us) server_negotiation.cc:206] Beginning negotiation
0811 09:09:10.786152 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:09:10.786180 (+ 28us) server_negotiation.cc:407] Connection header received
0811 09:09:10.786238 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:09:10.786244 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:09:10.786465 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:09:10.786622 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:09:10.787475 (+ 853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:09:10.787481 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:09:10.787492 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:09:10.787510 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:09:10.787551 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:09:10.787564 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:09:10.787681 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 09:09:10.787747 (+ 66us) server_negotiation.cc:299] Negotiation successful
0811 09:09:10.787837 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":250,"threads_started":1}
I20260811 09:09:10.789378 15534 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:09:10.871614 15541 heartbeater.cc:503] Master 127.28.154.190:33015 was elected leader, sending a full tablet report...
I20260811 09:09:10.873492 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 15335
I20260811 09:09:10.895756 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 15553
I20260811 09:09:10.899531 16069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:09:10.896801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40112 (local address 127.28.154.130:45277)
0811 09:09:10.897331 (+ 530us) server_negotiation.cc:206] Beginning negotiation
0811 09:09:10.897338 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:09:10.897365 (+ 27us) server_negotiation.cc:407] Connection header received
0811 09:09:10.897429 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:09:10.897436 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:09:10.897673 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:09:10.897827 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:09:10.898807 (+ 980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:09:10.898814 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:09:10.898825 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:09:10.898844 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:09:10.898885 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:09:10.898898 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:09:10.899030 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 09:09:10.899156 (+ 126us) server_negotiation.cc:299] Negotiation successful
0811 09:09:10.899263 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":391,"thread_start_us":261,"threads_started":1}
I20260811 09:09:10.900830 15785 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:09:10.985462 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 15553
I20260811 09:09:11.012197 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 15799
I20260811 09:09:11.015866 16071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:09:11.013133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36790 (local address 127.28.154.131:33073)
0811 09:09:11.013785 (+ 652us) server_negotiation.cc:206] Beginning negotiation
0811 09:09:11.013792 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:09:11.013826 (+ 34us) server_negotiation.cc:407] Connection header received
0811 09:09:11.013886 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:09:11.013892 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:09:11.014124 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:09:11.014256 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:09:11.015146 (+ 890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:09:11.015153 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:09:11.015164 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:09:11.015178 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:09:11.015223 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:09:11.015241 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:09:11.015375 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0811 09:09:11.015483 (+ 108us) server_negotiation.cc:299] Negotiation successful
0811 09:09:11.015575 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":464,"thread_start_us":285,"threads_started":1}
I20260811 09:09:11.017194 16037 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:09:11.098765 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 15799
I20260811 09:09:11.125761 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 15146
I20260811 09:09:11.127125 15273 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:09:11.230227 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 15146
2026-08-11T09:09:11Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2443 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439250192320-29290-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:09:11 dist-test-slave-xt4k krb5kdc[16082](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:09:11 dist-test-slave-xt4k krb5kdc[16082](info): set up 2 sockets
Aug 11 09:09:11 dist-test-slave-xt4k krb5kdc[16082](info): commencing operation
krb5kdc: starting...
W20260811 09:09:12.538357 29290 mini_kdc.cc:121] Time spent starting KDC: real 1.259s user 0.001s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439250192320-29290-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439250192320-29290-0/krb5kdc/test-user.keytab.
Aug 11 09:09:12 dist-test-slave-xt4k krb5kdc[16082](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439352, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:09:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:09:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.154.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.154.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.154.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab.
Entry for principal kudu/127.28.154.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab.
I20260811 09:09:12.837287 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439250192320-29290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439250192320-29290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439250192320-29290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.154.190:36123
--webserver_interface=127.28.154.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab
--principal=kudu/127.28.154.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:43137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.154.190:36123
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:09:13.180848 16106 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:09:13.181177 16106 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:09:13.181309 16106 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:09:13.188690 16106 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:09:13.188823 16106 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:09:13.188915 16106 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:09:13.188990 16106 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:09:13.189050 16106 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:09:13.199141 16106 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:43137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439250192320-29290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439250192320-29290-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.154.190:36123
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab
--principal=kudu/127.28.154.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.154.190:36123
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.154.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439250192320-29290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:09:13.200624 16106 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:09:13.201992 16106 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:09:13.213428 16115 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:09:13.213490 16120 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:09:13.213526 16117 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:09:13.214484 16106 server_base.cc:1034] running on GCE node
I20260811 09:09:13.215273 16106 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:09:13.216760 16106 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:09:13.217979 16106 hybrid_clock.cc:648] HybridClock initialized: now 1786439353217925 us; error 46 us; skew 500 ppm
I20260811 09:09:13.218319 16106 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:09:13 dist-test-slave-xt4k krb5kdc[16082](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439353, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:09:13.224876 16106 init.cc:377] Logged in from keytab as kudu/127.28.154.190@KRBTEST.COM (short username kudu)
I20260811 09:09:13.226529 16106 webserver.cc:460] Webserver started at http://127.28.154.190:39643/ using document root <none> and password file <none>
I20260811 09:09:13.227260 16106 fs_manager.cc:359] Metadata directory not provided
I20260811 09:09:13.227386 16106 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:09:13.227705 16106 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:09:13.230753 16106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439250192320-29290-0/minicluster-data/master-0/data/instance:
uuid: "bc29c652a824432d9ab3d7be4b5b624e"
format_stamp: "Formatted at 2026-08-11 09:09:13 on dist-test-slave-xt4k"
server_key: "f5c3df4aca235cc0cda76dc5689e9bba"
server_key_iv: "4f77cee29e43c211c02cbf5dbc801bf5"
server_key_version: "encryptionkey@0"
I20260811 09:09:13.231509 16106 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439250192320-29290-0/minicluster-data/master-0/wal/instance:
uuid: "bc29c652a824432d9ab3d7be4b5b624e"
format_stamp: "Formatted at 2026-08-11 09:09:13 on dist-test-slave-xt4k"
server_key: "f5c3df4aca235cc0cda76dc5689e9bba"
server_key_iv: "4f77cee29e43c211c02cbf5dbc801bf5"
server_key_version: "encryptionkey@0"
I20260811 09:09:13.237237 16106 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 09:09:13.240952 16132 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:09:13.242458 16106 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260811 09:09:13.242645 16106 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439250192320-29290-0/minicluster-data/master-0/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439250192320-29290-0/minicluster-data/master-0/wal
uuid: "bc29c652a824432d9ab3d7be4b5b624e"
format_stamp: "Formatted at 2026-08-11 09:09:13 on dist-test-slave-xt4k"
server_key: "f5c3df4aca235cc0cda76dc5689e9bba"
server_key_iv: "4f77cee29e43c211c02cbf5dbc801bf5"
server_key_version: "encryptionkey@0"
I20260811 09:09:13.242843 16106 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439250192320-29290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439250192320-29290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439250192320-29290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:09:13.260152 16106 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:09:13.264678 16106 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:09:13.264981 16106 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:09:13.283604 16106 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.190:36123
I20260811 09:09:13.283685 16230 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.190:36123 every 8 connection(s)
I20260811 09:09:13.285085 16106 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
I20260811 09:09:13.286810 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 16106
I20260811 09:09:13.287029 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439250192320-29290-0/minicluster-data/master-0/wal/instance
I20260811 09:09:13.287845 29290 external_mini_cluster.cc:1428] Setting key dfe9f560e00976eae78d47ef42b4b190
I20260811 09:09:13.290118 16233 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 09:09:13 dist-test-slave-xt4k krb5kdc[16082](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439352, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.154.190@KRBTEST.COM
I20260811 09:09:13.304437 16233 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc29c652a824432d9ab3d7be4b5b624e: Bootstrap starting.
I20260811 09:09:13.307832 16233 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bc29c652a824432d9ab3d7be4b5b624e: Neither blocks nor log segments found. Creating new log.
I20260811 09:09:13.309198 16233 log.cc:824] T 00000000000000000000000000000000 P bc29c652a824432d9ab3d7be4b5b624e: Log is configured to *not* fsync() on all Append() calls
I20260811 09:09:13.312799 16239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:09:13.290154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54658 (local address 127.28.154.190:36123)
0811 09:09:13.291066 (+ 912us) server_negotiation.cc:206] Beginning negotiation
0811 09:09:13.291079 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 09:09:13.291124 (+ 45us) server_negotiation.cc:407] Connection header received
0811 09:09:13.293008 (+ 1884us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:09:13.293035 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:09:13.293475 (+ 440us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:09:13.293949 (+ 474us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:09:13.296017 (+ 2068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:09:13.297526 (+ 1509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:09:13.298816 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:09:13.299514 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:09:13.299702 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:09:13.304567 (+ 4865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:09:13.304578 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:09:13.304599 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:09:13.304655 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:09:13.307800 (+ 3145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:09:13.308767 (+ 967us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:09:13.308775 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:09:13.308782 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:09:13.308973 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:09:13.309446 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:09:13.309453 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:09:13.309457 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:09:13.310083 (+ 626us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:09:13.310363 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0811 09:09:13.310971 (+ 608us) server_negotiation.cc:299] Negotiation successful
0811 09:09:13.311420 (+ 449us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":694,"thread_start_us":335,"threads_started":1}
I20260811 09:09:13.312960 16233 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc29c652a824432d9ab3d7be4b5b624e: No bootstrap required, opened a new log
I20260811 09:09:13.321053 16233 raft_consensus.cc:348] T 00000000000000000000000000000000 P bc29c652a824432d9ab3d7be4b5b624e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc29c652a824432d9ab3d7be4b5b624e" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 36123 } }
I20260811 09:09:13.321444 16233 raft_consensus.cc:374] T 00000000000000000000000000000000 P bc29c652a824432d9ab3d7be4b5b624e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:09:13.321524 16233 raft_consensus.cc:729] T 00000000000000000000000000000000 P bc29c652a824432d9ab3d7be4b5b624e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc29c652a824432d9ab3d7be4b5b624e, State: Initialized, Role: FOLLOWER
I20260811 09:09:13.322077 16233 consensus_queue.cc:260] T 00000000000000000000000000000000 P bc29c652a824432d9ab3d7be4b5b624e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc29c652a824432d9ab3d7be4b5b624e" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 36123 } }
I20260811 09:09:13.322288 16233 raft_consensus.cc:388] T 00000000000000000000000000000000 P bc29c652a824432d9ab3d7be4b5b624e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:09:13.322405 16233 raft_consensus.cc:482] T 00000000000000000000000000000000 P bc29c652a824432d9ab3d7be4b5b624e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:09:13.322535 16233 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bc29c652a824432d9ab3d7be4b5b624e [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:09:13.324827 16233 raft_consensus.cc:504] T 00000000000000000000000000000000 P bc29c652a824432d9ab3d7be4b5b624e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc29c652a824432d9ab3d7be4b5b624e" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 36123 } }
I20260811 09:09:13.325475 16233 leader_election.cc:302] T 00000000000000000000000000000000 P bc29c652a824432d9ab3d7be4b5b624e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bc29c652a824432d9ab3d7be4b5b624e; no voters:
I20260811 09:09:13.326093 16233 leader_election.cc:288] T 00000000000000000000000000000000 P bc29c652a824432d9ab3d7be4b5b624e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:09:13.326220 16242 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bc29c652a824432d9ab3d7be4b5b624e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:09:13.326913 16242 raft_consensus.cc:686] T 00000000000000000000000000000000 P bc29c652a824432d9ab3d7be4b5b624e [term 1 LEADER]: Becoming Leader. State: Replica: bc29c652a824432d9ab3d7be4b5b624e, State: Running, Role: LEADER
I20260811 09:09:13.327427 16242 consensus_queue.cc:237] T 00000000000000000000000000000000 P bc29c652a824432d9ab3d7be4b5b624e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc29c652a824432d9ab3d7be4b5b624e" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 36123 } }
I20260811 09:09:13.328092 16233 sys_catalog.cc:564] T 00000000000000000000000000000000 P bc29c652a824432d9ab3d7be4b5b624e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:09:13.330339 16243 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc29c652a824432d9ab3d7be4b5b624e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bc29c652a824432d9ab3d7be4b5b624e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc29c652a824432d9ab3d7be4b5b624e" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 36123 } } }
I20260811 09:09:13.330716 16243 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc29c652a824432d9ab3d7be4b5b624e [sys.catalog]: This master's current role is: LEADER
I20260811 09:09:13.330415 16245 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc29c652a824432d9ab3d7be4b5b624e [sys.catalog]: SysCatalogTable state changed. Reason: New leader bc29c652a824432d9ab3d7be4b5b624e. Latest consensus state: current_term: 1 leader_uuid: "bc29c652a824432d9ab3d7be4b5b624e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc29c652a824432d9ab3d7be4b5b624e" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 36123 } } }
I20260811 09:09:13.331108 16245 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc29c652a824432d9ab3d7be4b5b624e [sys.catalog]: This master's current role is: LEADER
I20260811 09:09:13.332875 16252 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:09:13.338552 16252 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:09:13.348256 16252 catalog_manager.cc:1347] Generated new cluster ID: afdd3df39bbb42d78b1ed8f5321be563
I20260811 09:09:13.348397 16252 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:09:13.366333 16252 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:09:13.367667 16252 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:09:13.383324 16252 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bc29c652a824432d9ab3d7be4b5b624e: Generated new TSK 0
I20260811 09:09:13.384518 16252 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.28.154.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.154.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.154.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.129.keytab.
Entry for principal kudu/127.28.154.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.129.keytab.
I20260811 09:09:13.460685 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439250192320-29290-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439250192320-29290-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439250192320-29290-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439250192320-29290-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.154.129:0
--local_ip_for_outbound_sockets=127.28.154.129
--webserver_interface=127.28.154.129
--webserver_port=0
--tserver_master_addrs=127.28.154.190:36123
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.129.keytab
--principal=kudu/127.28.154.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:43137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:09:13.809852 16278 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:09:13.810192 16278 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:09:13.810328 16278 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:09:13.817938 16278 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:09:13.818089 16278 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:09:13.818248 16278 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.154.129
I20260811 09:09:13.830009 16278 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:43137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439250192320-29290-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439250192320-29290-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.129.keytab
--principal=kudu/127.28.154.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.154.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439250192320-29290-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.154.129
--webserver_port=0
--tserver_master_addrs=127.28.154.190:36123
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.154.129
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439250192320-29290-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:09:13.831424 16278 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:09:13.832875 16278 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:09:13.846293 16290 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:09:13.846349 16294 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:09:13.846383 16298 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:09:13.847021 16278 server_base.cc:1034] running on GCE node
I20260811 09:09:13.848104 16278 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:09:13.849148 16278 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:09:13.850365 16278 hybrid_clock.cc:648] HybridClock initialized: now 1786439353850315 us; error 43 us; skew 500 ppm
I20260811 09:09:13.850841 16278 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:09:13 dist-test-slave-xt4k krb5kdc[16082](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439353, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:09:13.854148 16278 init.cc:377] Logged in from keytab as kudu/127.28.154.129@KRBTEST.COM (short username kudu)
I20260811 09:09:13.855690 16278 webserver.cc:460] Webserver started at http://127.28.154.129:41595/ using document root <none> and password file <none>
I20260811 09:09:13.856423 16278 fs_manager.cc:359] Metadata directory not provided
I20260811 09:09:13.856536 16278 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:09:13.856822 16278 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:09:13.860415 16278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439250192320-29290-0/minicluster-data/ts-0/data/instance:
uuid: "d7a6ebbeb61841ad933cf189b99e9b92"
format_stamp: "Formatted at 2026-08-11 09:09:13 on dist-test-slave-xt4k"
server_key: "f436c8d7454a8bcbbc11ab4e6b8c1ac2"
server_key_iv: "dac37c46537bc3004fc69ebf6b59a9b4"
server_key_version: "encryptionkey@0"
I20260811 09:09:13.861346 16278 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439250192320-29290-0/minicluster-data/ts-0/wal/instance:
uuid: "d7a6ebbeb61841ad933cf189b99e9b92"
format_stamp: "Formatted at 2026-08-11 09:09:13 on dist-test-slave-xt4k"
server_key: "f436c8d7454a8bcbbc11ab4e6b8c1ac2"
server_key_iv: "dac37c46537bc3004fc69ebf6b59a9b4"
server_key_version: "encryptionkey@0"
I20260811 09:09:13.866955 16278 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 09:09:13.871160 16313 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:09:13.873029 16278 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260811 09:09:13.873230 16278 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439250192320-29290-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439250192320-29290-0/minicluster-data/ts-0/wal
uuid: "d7a6ebbeb61841ad933cf189b99e9b92"
format_stamp: "Formatted at 2026-08-11 09:09:13 on dist-test-slave-xt4k"
server_key: "f436c8d7454a8bcbbc11ab4e6b8c1ac2"
server_key_iv: "dac37c46537bc3004fc69ebf6b59a9b4"
server_key_version: "encryptionkey@0"
I20260811 09:09:13.873449 16278 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439250192320-29290-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439250192320-29290-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439250192320-29290-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:09:13.900719 16278 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:09:13.905593 16278 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:09:13.905898 16278 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:09:13.906971 16278 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:09:13.908450 16278 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:09:13.908531 16278 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:09:13.908659 16278 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:09:13.908715 16278 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:09:13.941284 16278 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.129:38953
I20260811 09:09:13.941360 16532 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.129:38953 every 8 connection(s)
I20260811 09:09:13.942636 16278 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439250192320-29290-0/minicluster-data/ts-0/data/info.pb
I20260811 09:09:13.953115 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 16278
I20260811 09:09:13.953328 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439250192320-29290-0/minicluster-data/ts-0/wal/instance
I20260811 09:09:13.953996 29290 external_mini_cluster.cc:1428] Setting key de1ce2fd6f60a1e1963b816441a630e8
Aug 11 09:09:13 dist-test-slave-xt4k krb5kdc[16082](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439353, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.129@KRBTEST.COM for kudu/127.28.154.190@KRBTEST.COM
I20260811 09:09:13.963045 16536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:09:13.945547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.129:50989 (local address 127.28.154.190:36123)
0811 09:09:13.945982 (+ 435us) server_negotiation.cc:206] Beginning negotiation
0811 09:09:13.945986 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:09:13.946926 (+ 940us) server_negotiation.cc:407] Connection header received
0811 09:09:13.948857 (+ 1931us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:09:13.948863 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:09:13.948946 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:09:13.949043 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:09:13.950587 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:09:13.951303 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:09:13.952963 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:09:13.953399 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:09:13.953540 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:09:13.957291 (+ 3751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:09:13.957296 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:09:13.957299 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:09:13.957335 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:09:13.959810 (+ 2475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:09:13.960641 (+ 831us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:09:13.960646 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:09:13.960649 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:09:13.960771 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:09:13.961210 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:09:13.961213 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:09:13.961216 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:09:13.961497 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:09:13.961680 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0811 09:09:13.962445 (+ 765us) server_negotiation.cc:299] Negotiation successful
0811 09:09:13.962696 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327,"thread_start_us":221,"threads_started":1}
I20260811 09:09:13.964169 16537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:09:13.945972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:36123 (local address 127.28.154.129:50989)
0811 09:09:13.946732 (+ 760us) negotiation.cc:107] Waiting for socket to connect
0811 09:09:13.946778 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 09:09:13.948590 (+ 1812us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:09:13.949285 (+ 695us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:09:13.949294 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:09:13.949813 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:09:13.950404 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:09:13.950422 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:09:13.951468 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:09:13.951472 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:09:13.952804 (+ 1332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:09:13.952818 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:09:13.953634 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:09:13.953642 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:09:13.953879 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:09:13.954621 (+ 742us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:09:13.954653 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:09:13.957064 (+ 2411us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:09:13.959977 (+ 2913us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:09:13.959987 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:09:13.960003 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:09:13.960514 (+ 511us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:09:13.960878 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:09:13.960883 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:09:13.960887 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:09:13.961090 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:09:13.961671 (+ 581us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:09:13.961679 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:09:13.962028 (+ 349us) client_negotiation.cc:769] Sending connection context
0811 09:09:13.962288 (+ 260us) client_negotiation.cc:240] Negotiation successful
0811 09:09:13.962854 (+ 566us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":527,"thread_start_us":289,"threads_started":1}
I20260811 09:09:13.965960 16533 heartbeater.cc:347] Connected to a master server at 127.28.154.190:36123
I20260811 09:09:13.966281 16533 heartbeater.cc:464] Registering TS with master...
I20260811 09:09:13.967077 16533 heartbeater.cc:511] Master 127.28.154.190:36123 requested a full tablet report, sending...
I20260811 09:09:13.969625 16164 ts_manager.cc:194] Registered new tserver with Master: d7a6ebbeb61841ad933cf189b99e9b92 (127.28.154.129:38953)
I20260811 09:09:13.971661 16164 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.154.129@KRBTEST.COM'} at 127.28.154.129:50989
WARNING: no policy specified for kudu/127.28.154.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.154.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.154.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.130.keytab.
Entry for principal kudu/127.28.154.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.130.keytab.
I20260811 09:09:14.020503 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439250192320-29290-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439250192320-29290-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439250192320-29290-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439250192320-29290-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.154.130:0
--local_ip_for_outbound_sockets=127.28.154.130
--webserver_interface=127.28.154.130
--webserver_port=0
--tserver_master_addrs=127.28.154.190:36123
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.130.keytab
--principal=kudu/127.28.154.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:43137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:09:14.368450 16540 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:09:14.368764 16540 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:09:14.368894 16540 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:09:14.376360 16540 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:09:14.376488 16540 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:09:14.376626 16540 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.154.130
I20260811 09:09:14.386828 16540 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:43137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439250192320-29290-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439250192320-29290-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.130.keytab
--principal=kudu/127.28.154.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.154.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439250192320-29290-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.154.130
--webserver_port=0
--tserver_master_addrs=127.28.154.190:36123
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.154.130
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439250192320-29290-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:09:14.388150 16540 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:09:14.389604 16540 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:09:14.403761 16554 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:09:14.403915 16551 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:09:14.403859 16552 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:09:14.404969 16540 server_base.cc:1034] running on GCE node
I20260811 09:09:14.405651 16540 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:09:14.406690 16540 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:09:14.407914 16540 hybrid_clock.cc:648] HybridClock initialized: now 1786439354407837 us; error 65 us; skew 500 ppm
I20260811 09:09:14.408296 16540 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:09:14 dist-test-slave-xt4k krb5kdc[16082](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439354, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:09:14.411908 16540 init.cc:377] Logged in from keytab as kudu/127.28.154.130@KRBTEST.COM (short username kudu)
I20260811 09:09:14.414657 16540 webserver.cc:460] Webserver started at http://127.28.154.130:39797/ using document root <none> and password file <none>
I20260811 09:09:14.415405 16540 fs_manager.cc:359] Metadata directory not provided
I20260811 09:09:14.415539 16540 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:09:14.415819 16540 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:09:14.419348 16540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439250192320-29290-0/minicluster-data/ts-1/data/instance:
uuid: "c22db3988928400f86c90781e7e33d12"
format_stamp: "Formatted at 2026-08-11 09:09:14 on dist-test-slave-xt4k"
server_key: "c3a42e49f8862e07a80b63a0013d44cb"
server_key_iv: "e6ba8f3e5fcc7af80c555a3a4f4b67e1"
server_key_version: "encryptionkey@0"
I20260811 09:09:14.420173 16540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439250192320-29290-0/minicluster-data/ts-1/wal/instance:
uuid: "c22db3988928400f86c90781e7e33d12"
format_stamp: "Formatted at 2026-08-11 09:09:14 on dist-test-slave-xt4k"
server_key: "c3a42e49f8862e07a80b63a0013d44cb"
server_key_iv: "e6ba8f3e5fcc7af80c555a3a4f4b67e1"
server_key_version: "encryptionkey@0"
I20260811 09:09:14.425730 16540 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260811 09:09:14.429589 16574 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:09:14.431519 16540 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 09:09:14.431882 16540 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439250192320-29290-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439250192320-29290-0/minicluster-data/ts-1/wal
uuid: "c22db3988928400f86c90781e7e33d12"
format_stamp: "Formatted at 2026-08-11 09:09:14 on dist-test-slave-xt4k"
server_key: "c3a42e49f8862e07a80b63a0013d44cb"
server_key_iv: "e6ba8f3e5fcc7af80c555a3a4f4b67e1"
server_key_version: "encryptionkey@0"
I20260811 09:09:14.432132 16540 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439250192320-29290-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439250192320-29290-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439250192320-29290-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:09:14.463193 16540 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:09:14.468267 16540 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:09:14.468636 16540 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:09:14.469692 16540 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:09:14.471222 16540 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:09:14.471300 16540 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:09:14.471390 16540 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:09:14.471442 16540 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:09:14.503268 16540 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.130:43621
I20260811 09:09:14.503316 16772 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.130:43621 every 8 connection(s)
I20260811 09:09:14.504599 16540 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439250192320-29290-0/minicluster-data/ts-1/data/info.pb
I20260811 09:09:14.512040 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 16540
I20260811 09:09:14.512250 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439250192320-29290-0/minicluster-data/ts-1/wal/instance
I20260811 09:09:14.512948 29290 external_mini_cluster.cc:1428] Setting key e98e0463d2ac042d8221498a2b176ee1
Aug 11 09:09:14 dist-test-slave-xt4k krb5kdc[16082](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439354, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.130@KRBTEST.COM for kudu/127.28.154.190@KRBTEST.COM
I20260811 09:09:14.526414 16778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:09:14.507499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.130:51525 (local address 127.28.154.190:36123)
0811 09:09:14.508016 (+ 517us) server_negotiation.cc:206] Beginning negotiation
0811 09:09:14.508021 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:09:14.508969 (+ 948us) server_negotiation.cc:407] Connection header received
0811 09:09:14.510799 (+ 1830us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:09:14.510805 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:09:14.510886 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:09:14.510983 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:09:14.512989 (+ 2006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:09:14.513829 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:09:14.515838 (+ 2009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:09:14.516287 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:09:14.516431 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:09:14.520449 (+ 4018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:09:14.520480 (+ 31us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:09:14.520484 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:09:14.520517 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:09:14.523025 (+ 2508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:09:14.523903 (+ 878us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:09:14.523907 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:09:14.523910 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:09:14.524082 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:09:14.524608 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:09:14.524612 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:09:14.524615 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:09:14.524932 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:09:14.525056 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0811 09:09:14.525912 (+ 856us) server_negotiation.cc:299] Negotiation successful
0811 09:09:14.526112 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":399,"thread_start_us":282,"threads_started":1}
I20260811 09:09:14.527542 16781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:09:14.508093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:36123 (local address 127.28.154.130:51525)
0811 09:09:14.508815 (+ 722us) negotiation.cc:107] Waiting for socket to connect
0811 09:09:14.508861 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 09:09:14.510522 (+ 1661us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:09:14.511242 (+ 720us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:09:14.511250 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:09:14.511851 (+ 601us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:09:14.512768 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:09:14.512804 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:09:14.513990 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:09:14.513994 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:09:14.515670 (+ 1676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:09:14.515687 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:09:14.516531 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:09:14.516538 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:09:14.516833 (+ 295us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:09:14.517565 (+ 732us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:09:14.517593 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:09:14.520195 (+ 2602us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:09:14.523180 (+ 2985us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:09:14.523188 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:09:14.523205 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:09:14.523740 (+ 535us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:09:14.524193 (+ 453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:09:14.524198 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:09:14.524201 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:09:14.524485 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:09:14.525057 (+ 572us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:09:14.525065 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:09:14.525495 (+ 430us) client_negotiation.cc:769] Sending connection context
0811 09:09:14.525787 (+ 292us) client_negotiation.cc:240] Negotiation successful
0811 09:09:14.526331 (+ 544us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":307,"threads_started":1}
I20260811 09:09:14.529044 16773 heartbeater.cc:347] Connected to a master server at 127.28.154.190:36123
I20260811 09:09:14.529371 16773 heartbeater.cc:464] Registering TS with master...
I20260811 09:09:14.530118 16773 heartbeater.cc:511] Master 127.28.154.190:36123 requested a full tablet report, sending...
I20260811 09:09:14.531710 16164 ts_manager.cc:194] Registered new tserver with Master: c22db3988928400f86c90781e7e33d12 (127.28.154.130:43621)
I20260811 09:09:14.532603 16164 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.154.130@KRBTEST.COM'} at 127.28.154.130:51525
WARNING: no policy specified for kudu/127.28.154.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.154.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.154.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.131.keytab.
Entry for principal kudu/127.28.154.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.131.keytab.
I20260811 09:09:14.575670 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439250192320-29290-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439250192320-29290-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439250192320-29290-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439250192320-29290-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.154.131:0
--local_ip_for_outbound_sockets=127.28.154.131
--webserver_interface=127.28.154.131
--webserver_port=0
--tserver_master_addrs=127.28.154.190:36123
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.131.keytab
--principal=kudu/127.28.154.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:43137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:09:14.919463 16788 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:09:14.919765 16788 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:09:14.919862 16788 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:09:14.927590 16788 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:09:14.927723 16788 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:09:14.927840 16788 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.154.131
I20260811 09:09:14.938802 16788 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:43137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439250192320-29290-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439250192320-29290-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.131.keytab
--principal=kudu/127.28.154.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.154.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439250192320-29290-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.154.131
--webserver_port=0
--tserver_master_addrs=127.28.154.190:36123
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.154.131
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439250192320-29290-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:09:14.940130 16788 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:09:14.941457 16788 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:09:14.954777 16799 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:09:14.954823 16802 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:09:14.954845 16809 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:09:14.955695 16788 server_base.cc:1034] running on GCE node
I20260811 09:09:14.956395 16788 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:09:14.957541 16788 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:09:14.958812 16788 hybrid_clock.cc:648] HybridClock initialized: now 1786439354958756 us; error 44 us; skew 500 ppm
I20260811 09:09:14.959162 16788 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:09:14 dist-test-slave-xt4k krb5kdc[16082](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439354, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:09:14.962512 16788 init.cc:377] Logged in from keytab as kudu/127.28.154.131@KRBTEST.COM (short username kudu)
I20260811 09:09:14.964056 16788 webserver.cc:460] Webserver started at http://127.28.154.131:45977/ using document root <none> and password file <none>
I20260811 09:09:14.964795 16788 fs_manager.cc:359] Metadata directory not provided
I20260811 09:09:14.964900 16788 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:09:14.965158 16788 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:09:14.968056 16788 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439250192320-29290-0/minicluster-data/ts-2/data/instance:
uuid: "584ed0821f4c4d078dd295a60ebc56c8"
format_stamp: "Formatted at 2026-08-11 09:09:14 on dist-test-slave-xt4k"
server_key: "7f3997fa2fb0d97b4ca6f8aa938b4777"
server_key_iv: "d5fa7238a3f096cf53a8b43f06c37fdc"
server_key_version: "encryptionkey@0"
I20260811 09:09:14.968858 16788 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439250192320-29290-0/minicluster-data/ts-2/wal/instance:
uuid: "584ed0821f4c4d078dd295a60ebc56c8"
format_stamp: "Formatted at 2026-08-11 09:09:14 on dist-test-slave-xt4k"
server_key: "7f3997fa2fb0d97b4ca6f8aa938b4777"
server_key_iv: "d5fa7238a3f096cf53a8b43f06c37fdc"
server_key_version: "encryptionkey@0"
I20260811 09:09:14.974143 16788 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 09:09:14.975114 16533 heartbeater.cc:503] Master 127.28.154.190:36123 was elected leader, sending a full tablet report...
I20260811 09:09:14.977720 16821 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:09:14.979142 16788 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 09:09:14.979290 16788 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439250192320-29290-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439250192320-29290-0/minicluster-data/ts-2/wal
uuid: "584ed0821f4c4d078dd295a60ebc56c8"
format_stamp: "Formatted at 2026-08-11 09:09:14 on dist-test-slave-xt4k"
server_key: "7f3997fa2fb0d97b4ca6f8aa938b4777"
server_key_iv: "d5fa7238a3f096cf53a8b43f06c37fdc"
server_key_version: "encryptionkey@0"
I20260811 09:09:14.979481 16788 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439250192320-29290-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439250192320-29290-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439250192320-29290-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:09:15.003692 16788 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:09:15.008315 16788 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:09:15.008671 16788 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:09:15.009706 16788 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:09:15.011164 16788 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:09:15.011243 16788 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:09:15.011359 16788 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:09:15.011413 16788 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:09:15.042346 16788 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.131:37375
I20260811 09:09:15.042415 17039 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.131:37375 every 8 connection(s)
I20260811 09:09:15.043684 16788 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439250192320-29290-0/minicluster-data/ts-2/data/info.pb
I20260811 09:09:15.044678 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 16788
I20260811 09:09:15.044857 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439250192320-29290-0/minicluster-data/ts-2/wal/instance
I20260811 09:09:15.045307 29290 external_mini_cluster.cc:1428] Setting key 5513bdd0059af351668cd280b9a16d5d
Aug 11 09:09:15 dist-test-slave-xt4k krb5kdc[16082](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439354, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.131@KRBTEST.COM for kudu/127.28.154.190@KRBTEST.COM
I20260811 09:09:15.064386 17043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:09:15.046427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.131:50411 (local address 127.28.154.190:36123)
0811 09:09:15.047469 (+ 1042us) server_negotiation.cc:206] Beginning negotiation
0811 09:09:15.047475 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:09:15.048237 (+ 762us) server_negotiation.cc:407] Connection header received
0811 09:09:15.050024 (+ 1787us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:09:15.050028 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:09:15.050110 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:09:15.050198 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:09:15.051698 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:09:15.052514 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:09:15.054051 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:09:15.054504 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:09:15.054631 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:09:15.058087 (+ 3456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:09:15.058092 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:09:15.058096 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:09:15.058133 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:09:15.061387 (+ 3254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:09:15.062155 (+ 768us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:09:15.062159 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:09:15.062161 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:09:15.062297 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:09:15.062772 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:09:15.062776 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:09:15.062778 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:09:15.063046 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:09:15.063173 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0811 09:09:15.063858 (+ 685us) server_negotiation.cc:299] Negotiation successful
0811 09:09:15.064037 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":920,"thread_start_us":804,"threads_started":1}
I20260811 09:09:15.065323 17044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:09:15.047169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:36123 (local address 127.28.154.131:50411)
0811 09:09:15.048018 (+ 849us) negotiation.cc:107] Waiting for socket to connect
0811 09:09:15.048072 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 09:09:15.049775 (+ 1703us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:09:15.050389 (+ 614us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:09:15.050397 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:09:15.050947 (+ 550us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:09:15.051512 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:09:15.051530 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:09:15.052657 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:09:15.052660 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:09:15.053911 (+ 1251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:09:15.053924 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:09:15.054721 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:09:15.054726 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:09:15.054930 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:09:15.055573 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:09:15.055597 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:09:15.057884 (+ 2287us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:09:15.061553 (+ 3669us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:09:15.061561 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:09:15.061574 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:09:15.062043 (+ 469us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:09:15.062408 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:09:15.062413 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:09:15.062416 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:09:15.062633 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:09:15.063159 (+ 526us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:09:15.063165 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:09:15.063519 (+ 354us) client_negotiation.cc:769] Sending connection context
0811 09:09:15.063759 (+ 240us) client_negotiation.cc:240] Negotiation successful
0811 09:09:15.064181 (+ 422us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":563,"thread_start_us":328,"threads_started":1}
I20260811 09:09:15.066825 17040 heartbeater.cc:347] Connected to a master server at 127.28.154.190:36123
I20260811 09:09:15.067154 17040 heartbeater.cc:464] Registering TS with master...
I20260811 09:09:15.067884 17040 heartbeater.cc:511] Master 127.28.154.190:36123 requested a full tablet report, sending...
I20260811 09:09:15.069377 16164 ts_manager.cc:194] Registered new tserver with Master: 584ed0821f4c4d078dd295a60ebc56c8 (127.28.154.131:37375)
I20260811 09:09:15.070214 16164 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.154.131@KRBTEST.COM'} at 127.28.154.131:50411
I20260811 09:09:15.073483 29290 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 09:09:15.090426 17043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:09:15.077880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45268 (local address 127.28.154.190:36123)
0811 09:09:15.078033 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 09:09:15.078038 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:09:15.078554 (+ 516us) server_negotiation.cc:407] Connection header received
0811 09:09:15.078706 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:09:15.078709 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:09:15.078786 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:09:15.078867 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:09:15.080300 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:09:15.081133 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:09:15.082466 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:09:15.082904 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:09:15.083022 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:09:15.085294 (+ 2272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:09:15.085299 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:09:15.085303 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:09:15.085337 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:09:15.087903 (+ 2566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:09:15.088667 (+ 764us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:09:15.088671 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:09:15.088673 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:09:15.088801 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:09:15.089196 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:09:15.089199 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:09:15.089201 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:09:15.089512 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:09:15.089707 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0811 09:09:15.090021 (+ 314us) server_negotiation.cc:299] Negotiation successful
0811 09:09:15.090197 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 09:09:15.094506 16164 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45268:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 09:09:15.101297 16164 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 09:09:15.135675 17076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:09:15.114432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35458 (local address 127.28.154.129:38953)
0811 09:09:15.127374 (+ 12942us) server_negotiation.cc:206] Beginning negotiation
0811 09:09:15.127384 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 09:09:15.127418 (+ 34us) server_negotiation.cc:407] Connection header received
0811 09:09:15.127528 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:09:15.127538 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:09:15.127812 (+ 274us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:09:15.128048 (+ 236us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:09:15.129170 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:09:15.130707 (+ 1537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:09:15.133043 (+ 2336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:09:15.134445 (+ 1402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:09:15.134659 (+ 214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:09:15.134865 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0811 09:09:15.135145 (+ 280us) server_negotiation.cc:299] Negotiation successful
0811 09:09:15.135330 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12747,"thread_start_us":12591,"threads_started":1}
I20260811 09:09:15.135713 17074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:09:15.114137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.129:38953 (local address 127.0.0.1:35458)
0811 09:09:15.126329 (+ 12192us) negotiation.cc:107] Waiting for socket to connect
0811 09:09:15.126385 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 09:09:15.126641 (+ 256us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:09:15.128382 (+ 1741us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:09:15.128389 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:09:15.128429 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:09:15.128950 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:09:15.128964 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:09:15.130838 (+ 1874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:09:15.130843 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:09:15.132839 (+ 1996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:09:15.132856 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:09:15.134644 (+ 1788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:09:15.134650 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:09:15.134855 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:09:15.134908 (+ 53us) client_negotiation.cc:769] Sending connection context
0811 09:09:15.135061 (+ 153us) client_negotiation.cc:240] Negotiation successful
0811 09:09:15.135277 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11864,"spinlock_wait_cycles":11520,"thread_start_us":14703,"threads_started":1}
I20260811 09:09:15.137030 17077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:09:15.114537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38484 (local address 127.28.154.130:43621)
0811 09:09:15.125393 (+ 10856us) server_negotiation.cc:206] Beginning negotiation
0811 09:09:15.125405 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 09:09:15.128489 (+ 3084us) server_negotiation.cc:407] Connection header received
0811 09:09:15.128712 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:09:15.128718 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:09:15.128945 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:09:15.129194 (+ 249us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:09:15.130144 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:09:15.131566 (+ 1422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:09:15.134147 (+ 2581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:09:15.135578 (+ 1431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:09:15.135746 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:09:15.136521 (+ 775us) server_negotiation.cc:1035] Waiting for connection context
0811 09:09:15.136585 (+ 64us) server_negotiation.cc:299] Negotiation successful
0811 09:09:15.136742 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10634,"thread_start_us":10887,"threads_started":1}
I20260811 09:09:15.137043 17078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:09:15.114290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.130:43621 (local address 127.0.0.1:38484)
0811 09:09:15.128325 (+ 14035us) negotiation.cc:107] Waiting for socket to connect
0811 09:09:15.128366 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 09:09:15.128529 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:09:15.129480 (+ 951us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:09:15.129484 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:09:15.129507 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:09:15.129909 (+ 402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:09:15.129924 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:09:15.131719 (+ 1795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:09:15.131725 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:09:15.133960 (+ 2235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:09:15.133987 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:09:15.136136 (+ 2149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:09:15.136143 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:09:15.136309 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:09:15.136362 (+ 53us) client_negotiation.cc:769] Sending connection context
0811 09:09:15.136445 (+ 83us) client_negotiation.cc:240] Negotiation successful
0811 09:09:15.136591 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13895,"mutex_wait_us":3,"thread_start_us":11548,"threads_started":1}
I20260811 09:09:15.139137 17072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:09:15.116713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.131:37375 (local address 127.0.0.1:56012)
0811 09:09:15.128659 (+ 11946us) negotiation.cc:107] Waiting for socket to connect
0811 09:09:15.128678 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 09:09:15.128807 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:09:15.129889 (+ 1082us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:09:15.129893 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:09:15.129920 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:09:15.130311 (+ 391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:09:15.130324 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:09:15.132457 (+ 2133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:09:15.132463 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:09:15.134375 (+ 1912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:09:15.134390 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:09:15.136982 (+ 2592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:09:15.136989 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:09:15.138523 (+ 1534us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:09:15.138558 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 09:09:15.138655 (+ 97us) client_negotiation.cc:240] Negotiation successful
0811 09:09:15.138786 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11810,"mutex_wait_us":25,"thread_start_us":11348,"threads_started":1}
I20260811 09:09:15.139696 17079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:09:15.117319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56012 (local address 127.28.154.131:37375)
0811 09:09:15.127857 (+ 10538us) server_negotiation.cc:206] Beginning negotiation
0811 09:09:15.127867 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 09:09:15.128794 (+ 927us) server_negotiation.cc:407] Connection header received
0811 09:09:15.129038 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:09:15.129047 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:09:15.129339 (+ 292us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:09:15.129586 (+ 247us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:09:15.130496 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:09:15.131989 (+ 1493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:09:15.134549 (+ 2560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:09:15.136358 (+ 1809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:09:15.136549 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:09:15.136706 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0811 09:09:15.138729 (+ 2023us) server_negotiation.cc:299] Negotiation successful
0811 09:09:15.138932 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10357,"thread_start_us":10189,"threads_started":1}
I20260811 09:09:15.140183 16402 tablet_service.cc:1467] Processing CreateTablet for tablet aa2294f93df5402eb9873c5e14f801ed (DEFAULT_TABLE table=test-table [id=662fa245f3b74fdb9e4da1e0c9841567]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:09:15.140947 16656 tablet_service.cc:1467] Processing CreateTablet for tablet aa2294f93df5402eb9873c5e14f801ed (DEFAULT_TABLE table=test-table [id=662fa245f3b74fdb9e4da1e0c9841567]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:09:15.141791 16402 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aa2294f93df5402eb9873c5e14f801ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:09:15.142336 16656 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aa2294f93df5402eb9873c5e14f801ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:09:15.143330 16910 tablet_service.cc:1467] Processing CreateTablet for tablet aa2294f93df5402eb9873c5e14f801ed (DEFAULT_TABLE table=test-table [id=662fa245f3b74fdb9e4da1e0c9841567]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:09:15.144846 16910 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aa2294f93df5402eb9873c5e14f801ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:09:15.154412 17081 tablet_bootstrap.cc:492] T aa2294f93df5402eb9873c5e14f801ed P c22db3988928400f86c90781e7e33d12: Bootstrap starting.
I20260811 09:09:15.156567 17082 tablet_bootstrap.cc:492] T aa2294f93df5402eb9873c5e14f801ed P d7a6ebbeb61841ad933cf189b99e9b92: Bootstrap starting.
I20260811 09:09:15.158627 17081 tablet_bootstrap.cc:654] T aa2294f93df5402eb9873c5e14f801ed P c22db3988928400f86c90781e7e33d12: Neither blocks nor log segments found. Creating new log.
I20260811 09:09:15.158946 17084 tablet_bootstrap.cc:492] T aa2294f93df5402eb9873c5e14f801ed P 584ed0821f4c4d078dd295a60ebc56c8: Bootstrap starting.
I20260811 09:09:15.160312 17082 tablet_bootstrap.cc:654] T aa2294f93df5402eb9873c5e14f801ed P d7a6ebbeb61841ad933cf189b99e9b92: Neither blocks nor log segments found. Creating new log.
I20260811 09:09:15.160671 17081 log.cc:824] T aa2294f93df5402eb9873c5e14f801ed P c22db3988928400f86c90781e7e33d12: Log is configured to *not* fsync() on all Append() calls
I20260811 09:09:15.161625 17082 log.cc:824] T aa2294f93df5402eb9873c5e14f801ed P d7a6ebbeb61841ad933cf189b99e9b92: Log is configured to *not* fsync() on all Append() calls
I20260811 09:09:15.162349 17084 tablet_bootstrap.cc:654] T aa2294f93df5402eb9873c5e14f801ed P 584ed0821f4c4d078dd295a60ebc56c8: Neither blocks nor log segments found. Creating new log.
I20260811 09:09:15.163277 17084 log.cc:824] T aa2294f93df5402eb9873c5e14f801ed P 584ed0821f4c4d078dd295a60ebc56c8: Log is configured to *not* fsync() on all Append() calls
I20260811 09:09:15.163841 17081 tablet_bootstrap.cc:492] T aa2294f93df5402eb9873c5e14f801ed P c22db3988928400f86c90781e7e33d12: No bootstrap required, opened a new log
I20260811 09:09:15.164147 17081 ts_tablet_manager.cc:1397] T aa2294f93df5402eb9873c5e14f801ed P c22db3988928400f86c90781e7e33d12: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.008s
I20260811 09:09:15.166193 17082 tablet_bootstrap.cc:492] T aa2294f93df5402eb9873c5e14f801ed P d7a6ebbeb61841ad933cf189b99e9b92: No bootstrap required, opened a new log
I20260811 09:09:15.166193 17084 tablet_bootstrap.cc:492] T aa2294f93df5402eb9873c5e14f801ed P 584ed0821f4c4d078dd295a60ebc56c8: No bootstrap required, opened a new log
I20260811 09:09:15.166482 17082 ts_tablet_manager.cc:1397] T aa2294f93df5402eb9873c5e14f801ed P d7a6ebbeb61841ad933cf189b99e9b92: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.003s
I20260811 09:09:15.166471 17084 ts_tablet_manager.cc:1397] T aa2294f93df5402eb9873c5e14f801ed P 584ed0821f4c4d078dd295a60ebc56c8: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.005s
I20260811 09:09:15.171864 17081 raft_consensus.cc:348] T aa2294f93df5402eb9873c5e14f801ed P c22db3988928400f86c90781e7e33d12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7a6ebbeb61841ad933cf189b99e9b92" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 38953 } } peers { permanent_uuid: "c22db3988928400f86c90781e7e33d12" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 43621 } } peers { permanent_uuid: "584ed0821f4c4d078dd295a60ebc56c8" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 37375 } }
I20260811 09:09:15.172255 17081 raft_consensus.cc:374] T aa2294f93df5402eb9873c5e14f801ed P c22db3988928400f86c90781e7e33d12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:09:15.172381 17081 raft_consensus.cc:729] T aa2294f93df5402eb9873c5e14f801ed P c22db3988928400f86c90781e7e33d12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c22db3988928400f86c90781e7e33d12, State: Initialized, Role: FOLLOWER
I20260811 09:09:15.172984 17081 consensus_queue.cc:260] T aa2294f93df5402eb9873c5e14f801ed P c22db3988928400f86c90781e7e33d12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7a6ebbeb61841ad933cf189b99e9b92" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 38953 } } peers { permanent_uuid: "c22db3988928400f86c90781e7e33d12" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 43621 } } peers { permanent_uuid: "584ed0821f4c4d078dd295a60ebc56c8" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 37375 } }
I20260811 09:09:15.174276 16773 heartbeater.cc:503] Master 127.28.154.190:36123 was elected leader, sending a full tablet report...
I20260811 09:09:15.175127 17081 ts_tablet_manager.cc:1428] T aa2294f93df5402eb9873c5e14f801ed P c22db3988928400f86c90781e7e33d12: Time spent starting tablet: real 0.011s user 0.010s sys 0.000s
I20260811 09:09:15.177178 17082 raft_consensus.cc:348] T aa2294f93df5402eb9873c5e14f801ed P d7a6ebbeb61841ad933cf189b99e9b92 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7a6ebbeb61841ad933cf189b99e9b92" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 38953 } } peers { permanent_uuid: "c22db3988928400f86c90781e7e33d12" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 43621 } } peers { permanent_uuid: "584ed0821f4c4d078dd295a60ebc56c8" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 37375 } }
I20260811 09:09:15.177184 17084 raft_consensus.cc:348] T aa2294f93df5402eb9873c5e14f801ed P 584ed0821f4c4d078dd295a60ebc56c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7a6ebbeb61841ad933cf189b99e9b92" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 38953 } } peers { permanent_uuid: "c22db3988928400f86c90781e7e33d12" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 43621 } } peers { permanent_uuid: "584ed0821f4c4d078dd295a60ebc56c8" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 37375 } }
I20260811 09:09:15.177657 17084 raft_consensus.cc:374] T aa2294f93df5402eb9873c5e14f801ed P 584ed0821f4c4d078dd295a60ebc56c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:09:15.177657 17082 raft_consensus.cc:374] T aa2294f93df5402eb9873c5e14f801ed P d7a6ebbeb61841ad933cf189b99e9b92 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:09:15.177763 17084 raft_consensus.cc:729] T aa2294f93df5402eb9873c5e14f801ed P 584ed0821f4c4d078dd295a60ebc56c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 584ed0821f4c4d078dd295a60ebc56c8, State: Initialized, Role: FOLLOWER
I20260811 09:09:15.177762 17082 raft_consensus.cc:729] T aa2294f93df5402eb9873c5e14f801ed P d7a6ebbeb61841ad933cf189b99e9b92 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7a6ebbeb61841ad933cf189b99e9b92, State: Initialized, Role: FOLLOWER
I20260811 09:09:15.178404 17082 consensus_queue.cc:260] T aa2294f93df5402eb9873c5e14f801ed P d7a6ebbeb61841ad933cf189b99e9b92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7a6ebbeb61841ad933cf189b99e9b92" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 38953 } } peers { permanent_uuid: "c22db3988928400f86c90781e7e33d12" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 43621 } } peers { permanent_uuid: "584ed0821f4c4d078dd295a60ebc56c8" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 37375 } }
I20260811 09:09:15.178459 17084 consensus_queue.cc:260] T aa2294f93df5402eb9873c5e14f801ed P 584ed0821f4c4d078dd295a60ebc56c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7a6ebbeb61841ad933cf189b99e9b92" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 38953 } } peers { permanent_uuid: "c22db3988928400f86c90781e7e33d12" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 43621 } } peers { permanent_uuid: "584ed0821f4c4d078dd295a60ebc56c8" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 37375 } }
I20260811 09:09:15.180315 17084 ts_tablet_manager.cc:1428] T aa2294f93df5402eb9873c5e14f801ed P 584ed0821f4c4d078dd295a60ebc56c8: Time spent starting tablet: real 0.014s user 0.010s sys 0.005s
I20260811 09:09:15.180655 17082 ts_tablet_manager.cc:1428] T aa2294f93df5402eb9873c5e14f801ed P d7a6ebbeb61841ad933cf189b99e9b92: Time spent starting tablet: real 0.014s user 0.015s sys 0.001s
I20260811 09:09:15.181456 17040 heartbeater.cc:503] Master 127.28.154.190:36123 was elected leader, sending a full tablet report...
W20260811 09:09:15.195232 16535 tablet.cc:2369] T aa2294f93df5402eb9873c5e14f801ed P d7a6ebbeb61841ad933cf189b99e9b92: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 09:09:15.257078 16776 tablet.cc:2369] T aa2294f93df5402eb9873c5e14f801ed P c22db3988928400f86c90781e7e33d12: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 09:09:15.295368 17042 tablet.cc:2369] T aa2294f93df5402eb9873c5e14f801ed P 584ed0821f4c4d078dd295a60ebc56c8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 09:09:15.441294 17090 raft_consensus.cc:482] T aa2294f93df5402eb9873c5e14f801ed P c22db3988928400f86c90781e7e33d12 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:09:15.441609 17090 raft_consensus.cc:504] T aa2294f93df5402eb9873c5e14f801ed P c22db3988928400f86c90781e7e33d12 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7a6ebbeb61841ad933cf189b99e9b92" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 38953 } } peers { permanent_uuid: "c22db3988928400f86c90781e7e33d12" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 43621 } } peers { permanent_uuid: "584ed0821f4c4d078dd295a60ebc56c8" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 37375 } }
I20260811 09:09:15.443423 17090 leader_election.cc:288] T aa2294f93df5402eb9873c5e14f801ed P c22db3988928400f86c90781e7e33d12 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d7a6ebbeb61841ad933cf189b99e9b92 (127.28.154.129:38953), 584ed0821f4c4d078dd295a60ebc56c8 (127.28.154.131:37375)
I20260811 09:09:15.452718 17079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:09:15.444014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.130:42553 (local address 127.28.154.131:37375)
0811 09:09:15.444232 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0811 09:09:15.444239 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:09:15.445183 (+ 944us) server_negotiation.cc:407] Connection header received
0811 09:09:15.445357 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:09:15.445361 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:09:15.445424 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:09:15.445512 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:09:15.446250 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:09:15.447311 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:09:15.450223 (+ 2912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:09:15.451468 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:09:15.451634 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:09:15.451712 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 09:09:15.452272 (+ 560us) server_negotiation.cc:299] Negotiation successful
0811 09:09:15.452444 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 09:09:15.452710 17076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:09:15.443910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.130:55257 (local address 127.28.154.129:38953)
0811 09:09:15.444153 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0811 09:09:15.444159 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:09:15.444962 (+ 803us) server_negotiation.cc:407] Connection header received
0811 09:09:15.445230 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:09:15.445236 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:09:15.445317 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:09:15.445443 (+ 126us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:09:15.446131 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:09:15.447326 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:09:15.450214 (+ 2888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:09:15.451458 (+ 1244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:09:15.451638 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:09:15.451712 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 09:09:15.452272 (+ 560us) server_negotiation.cc:299] Negotiation successful
0811 09:09:15.452444 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260811 09:09:15.452967 17100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:09:15.443660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.129:38953 (local address 127.28.154.130:55257)
0811 09:09:15.444841 (+ 1181us) negotiation.cc:107] Waiting for socket to connect
0811 09:09:15.444859 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 09:09:15.445001 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:09:15.445489 (+ 488us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:09:15.445495 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:09:15.445519 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:09:15.445983 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:09:15.445998 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:09:15.447493 (+ 1495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:09:15.447499 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:09:15.450027 (+ 2528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:09:15.450041 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:09:15.451761 (+ 1720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:09:15.451768 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:09:15.452052 (+ 284us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:09:15.452110 (+ 58us) client_negotiation.cc:769] Sending connection context
0811 09:09:15.452208 (+ 98us) client_negotiation.cc:240] Negotiation successful
0811 09:09:15.452467 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1056,"mutex_wait_us":288,"thread_start_us":853,"threads_started":1}
I20260811 09:09:15.453259 17096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:09:15.443896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.131:37375 (local address 127.28.154.130:42553)
0811 09:09:15.445026 (+ 1130us) negotiation.cc:107] Waiting for socket to connect
0811 09:09:15.445042 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 09:09:15.445244 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:09:15.445543 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:09:15.445548 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:09:15.445578 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:09:15.446057 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:09:15.446069 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:09:15.447495 (+ 1426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:09:15.447499 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:09:15.450026 (+ 2527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:09:15.450041 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:09:15.451760 (+ 1719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:09:15.451770 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:09:15.452040 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:09:15.452105 (+ 65us) client_negotiation.cc:769] Sending connection context
0811 09:09:15.452218 (+ 113us) client_negotiation.cc:240] Negotiation successful
0811 09:09:15.452467 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1003,"mutex_wait_us":202,"thread_start_us":483,"threads_started":1}
I20260811 09:09:15.454253 16950 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa2294f93df5402eb9873c5e14f801ed" candidate_uuid: "c22db3988928400f86c90781e7e33d12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "584ed0821f4c4d078dd295a60ebc56c8" is_pre_election: true
I20260811 09:09:15.454253 16432 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa2294f93df5402eb9873c5e14f801ed" candidate_uuid: "c22db3988928400f86c90781e7e33d12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7a6ebbeb61841ad933cf189b99e9b92" is_pre_election: true
I20260811 09:09:15.454865 16432 raft_consensus.cc:2454] T aa2294f93df5402eb9873c5e14f801ed P d7a6ebbeb61841ad933cf189b99e9b92 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c22db3988928400f86c90781e7e33d12 in term 0.
I20260811 09:09:15.454864 16950 raft_consensus.cc:2454] T aa2294f93df5402eb9873c5e14f801ed P 584ed0821f4c4d078dd295a60ebc56c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c22db3988928400f86c90781e7e33d12 in term 0.
I20260811 09:09:15.455685 16575 leader_election.cc:302] T aa2294f93df5402eb9873c5e14f801ed P c22db3988928400f86c90781e7e33d12 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 584ed0821f4c4d078dd295a60ebc56c8, c22db3988928400f86c90781e7e33d12; no voters:
I20260811 09:09:15.456105 17090 raft_consensus.cc:2781] T aa2294f93df5402eb9873c5e14f801ed P c22db3988928400f86c90781e7e33d12 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 09:09:15.456260 17090 raft_consensus.cc:482] T aa2294f93df5402eb9873c5e14f801ed P c22db3988928400f86c90781e7e33d12 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 09:09:15.456391 17090 raft_consensus.cc:3037] T aa2294f93df5402eb9873c5e14f801ed P c22db3988928400f86c90781e7e33d12 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:09:15.458729 17090 raft_consensus.cc:504] T aa2294f93df5402eb9873c5e14f801ed P c22db3988928400f86c90781e7e33d12 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7a6ebbeb61841ad933cf189b99e9b92" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 38953 } } peers { permanent_uuid: "c22db3988928400f86c90781e7e33d12" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 43621 } } peers { permanent_uuid: "584ed0821f4c4d078dd295a60ebc56c8" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 37375 } }
I20260811 09:09:15.459348 17090 leader_election.cc:288] T aa2294f93df5402eb9873c5e14f801ed P c22db3988928400f86c90781e7e33d12 [CANDIDATE]: Term 1 election: Requested vote from peers d7a6ebbeb61841ad933cf189b99e9b92 (127.28.154.129:38953), 584ed0821f4c4d078dd295a60ebc56c8 (127.28.154.131:37375)
I20260811 09:09:15.459950 16432 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa2294f93df5402eb9873c5e14f801ed" candidate_uuid: "c22db3988928400f86c90781e7e33d12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7a6ebbeb61841ad933cf189b99e9b92"
I20260811 09:09:15.460042 16950 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa2294f93df5402eb9873c5e14f801ed" candidate_uuid: "c22db3988928400f86c90781e7e33d12" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "584ed0821f4c4d078dd295a60ebc56c8"
I20260811 09:09:15.460158 16432 raft_consensus.cc:3037] T aa2294f93df5402eb9873c5e14f801ed P d7a6ebbeb61841ad933cf189b99e9b92 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:09:15.460258 16950 raft_consensus.cc:3037] T aa2294f93df5402eb9873c5e14f801ed P 584ed0821f4c4d078dd295a60ebc56c8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:09:15.462518 16950 raft_consensus.cc:2454] T aa2294f93df5402eb9873c5e14f801ed P 584ed0821f4c4d078dd295a60ebc56c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c22db3988928400f86c90781e7e33d12 in term 1.
I20260811 09:09:15.462525 16432 raft_consensus.cc:2454] T aa2294f93df5402eb9873c5e14f801ed P d7a6ebbeb61841ad933cf189b99e9b92 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c22db3988928400f86c90781e7e33d12 in term 1.
I20260811 09:09:15.462997 16575 leader_election.cc:302] T aa2294f93df5402eb9873c5e14f801ed P c22db3988928400f86c90781e7e33d12 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 584ed0821f4c4d078dd295a60ebc56c8, c22db3988928400f86c90781e7e33d12; no voters:
I20260811 09:09:15.463332 17090 raft_consensus.cc:2781] T aa2294f93df5402eb9873c5e14f801ed P c22db3988928400f86c90781e7e33d12 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:09:15.463804 17090 raft_consensus.cc:686] T aa2294f93df5402eb9873c5e14f801ed P c22db3988928400f86c90781e7e33d12 [term 1 LEADER]: Becoming Leader. State: Replica: c22db3988928400f86c90781e7e33d12, State: Running, Role: LEADER
I20260811 09:09:15.464397 17090 consensus_queue.cc:237] T aa2294f93df5402eb9873c5e14f801ed P c22db3988928400f86c90781e7e33d12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7a6ebbeb61841ad933cf189b99e9b92" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 38953 } } peers { permanent_uuid: "c22db3988928400f86c90781e7e33d12" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 43621 } } peers { permanent_uuid: "584ed0821f4c4d078dd295a60ebc56c8" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 37375 } }
I20260811 09:09:15.469211 16162 catalog_manager.cc:5495] T aa2294f93df5402eb9873c5e14f801ed P c22db3988928400f86c90781e7e33d12 reported cstate change: term changed from 0 to 1, leader changed from <none> to c22db3988928400f86c90781e7e33d12 (127.28.154.130). New cstate: current_term: 1 leader_uuid: "c22db3988928400f86c90781e7e33d12" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d7a6ebbeb61841ad933cf189b99e9b92" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 38953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c22db3988928400f86c90781e7e33d12" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 43621 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "584ed0821f4c4d078dd295a60ebc56c8" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 37375 } health_report { overall_health: UNKNOWN } } }
I20260811 09:09:15.511502 17077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:09:15.505178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38494 (local address 127.28.154.130:43621)
0811 09:09:15.505384 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0811 09:09:15.505391 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:09:15.505528 (+ 137us) server_negotiation.cc:407] Connection header received
0811 09:09:15.505697 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:09:15.505701 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:09:15.505790 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:09:15.505914 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:09:15.506333 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:09:15.507096 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:09:15.508972 (+ 1876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:09:15.509451 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:09:15.509556 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:09:15.510128 (+ 572us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:09:15.510772 (+ 644us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:09:15.510910 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0811 09:09:15.511083 (+ 173us) server_negotiation.cc:299] Negotiation successful
0811 09:09:15.511171 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 09:09:15.520684 16950 raft_consensus.cc:1261] T aa2294f93df5402eb9873c5e14f801ed P 584ed0821f4c4d078dd295a60ebc56c8 [term 1 FOLLOWER]: Refusing update from remote peer c22db3988928400f86c90781e7e33d12: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:09:15.520684 16432 raft_consensus.cc:1261] T aa2294f93df5402eb9873c5e14f801ed P d7a6ebbeb61841ad933cf189b99e9b92 [term 1 FOLLOWER]: Refusing update from remote peer c22db3988928400f86c90781e7e33d12: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:09:15.521663 17101 consensus_queue.cc:1035] T aa2294f93df5402eb9873c5e14f801ed P c22db3988928400f86c90781e7e33d12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "584ed0821f4c4d078dd295a60ebc56c8" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 37375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:09:15.522064 17090 consensus_queue.cc:1035] T aa2294f93df5402eb9873c5e14f801ed P c22db3988928400f86c90781e7e33d12 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7a6ebbeb61841ad933cf189b99e9b92" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 38953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:09:15.533165 17106 mvcc.cc:205] Tried to move back new op lower bound from 7317255600201711616 to 7317255599986573312. Current Snapshot: MvccSnapshot[applied={T|T < 7317255600201711616}]
I20260811 09:09:15.547546 16162 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45268:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:09:15.547821 16162 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45268:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:09:15.551432 16162 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bc29c652a824432d9ab3d7be4b5b624e: Sending DeleteTablet for 3 replicas of tablet aa2294f93df5402eb9873c5e14f801ed
I20260811 09:09:15.552616 16656 tablet_service.cc:1514] Processing DeleteTablet for tablet aa2294f93df5402eb9873c5e14f801ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:09:15 UTC) from {username='kudu', principal='kudu/127.28.154.190@KRBTEST.COM'} at 127.0.0.1:38484
I20260811 09:09:15.552632 16910 tablet_service.cc:1514] Processing DeleteTablet for tablet aa2294f93df5402eb9873c5e14f801ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:09:15 UTC) from {username='kudu', principal='kudu/127.28.154.190@KRBTEST.COM'} at 127.0.0.1:56012
I20260811 09:09:15.552632 16402 tablet_service.cc:1514] Processing DeleteTablet for tablet aa2294f93df5402eb9873c5e14f801ed with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:09:15 UTC) from {username='kudu', principal='kudu/127.28.154.190@KRBTEST.COM'} at 127.0.0.1:35458
I20260811 09:09:15.554965 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 16278
I20260811 09:09:15.565991 17134 tablet_replica.cc:330] T aa2294f93df5402eb9873c5e14f801ed P d7a6ebbeb61841ad933cf189b99e9b92: stopping tablet replica
I20260811 09:09:15.566592 17134 raft_consensus.cc:2229] T aa2294f93df5402eb9873c5e14f801ed P d7a6ebbeb61841ad933cf189b99e9b92 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:09:15.567216 17134 raft_consensus.cc:2258] T aa2294f93df5402eb9873c5e14f801ed P d7a6ebbeb61841ad933cf189b99e9b92 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:09:15.568315 17135 tablet_replica.cc:330] T aa2294f93df5402eb9873c5e14f801ed P 584ed0821f4c4d078dd295a60ebc56c8: stopping tablet replica
I20260811 09:09:15.568691 17135 raft_consensus.cc:2229] T aa2294f93df5402eb9873c5e14f801ed P 584ed0821f4c4d078dd295a60ebc56c8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:09:15.569090 17135 raft_consensus.cc:2258] T aa2294f93df5402eb9873c5e14f801ed P 584ed0821f4c4d078dd295a60ebc56c8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:09:15.570509 17134 ts_tablet_manager.cc:1905] T aa2294f93df5402eb9873c5e14f801ed P d7a6ebbeb61841ad933cf189b99e9b92: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:09:15.574851 17138 tablet_replica.cc:330] T aa2294f93df5402eb9873c5e14f801ed P c22db3988928400f86c90781e7e33d12: stopping tablet replica
I20260811 09:09:15.575376 17138 raft_consensus.cc:2229] T aa2294f93df5402eb9873c5e14f801ed P c22db3988928400f86c90781e7e33d12 [term 1 LEADER]: Raft consensus shutting down.
I20260811 09:09:15.576064 17138 raft_consensus.cc:2258] T aa2294f93df5402eb9873c5e14f801ed P c22db3988928400f86c90781e7e33d12 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:09:15.578745 17138 ts_tablet_manager.cc:1905] T aa2294f93df5402eb9873c5e14f801ed P c22db3988928400f86c90781e7e33d12: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:09:15.578811 17135 ts_tablet_manager.cc:1905] T aa2294f93df5402eb9873c5e14f801ed P 584ed0821f4c4d078dd295a60ebc56c8: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 11 09:09:15 dist-test-slave-xt4k krb5kdc[16082](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439352, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.154.129@KRBTEST.COM
I20260811 09:09:15.586647 17138 ts_tablet_manager.cc:1918] T aa2294f93df5402eb9873c5e14f801ed P c22db3988928400f86c90781e7e33d12: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:09:15.586874 17138 log.cc:1196] T aa2294f93df5402eb9873c5e14f801ed P c22db3988928400f86c90781e7e33d12: Deleting WAL directory at /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439250192320-29290-0/minicluster-data/ts-1/wal/wals/aa2294f93df5402eb9873c5e14f801ed
I20260811 09:09:15.587447 17138 ts_tablet_manager.cc:1939] T aa2294f93df5402eb9873c5e14f801ed P c22db3988928400f86c90781e7e33d12: Deleting consensus metadata
I20260811 09:09:15.588877 17076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:09:15.556671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35470 (local address 127.28.154.129:38953)
0811 09:09:15.557294 (+ 623us) server_negotiation.cc:206] Beginning negotiation
0811 09:09:15.557301 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:09:15.561454 (+ 4153us) server_negotiation.cc:407] Connection header received
0811 09:09:15.561664 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:09:15.561670 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:09:15.561769 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:09:15.561876 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:09:15.570921 (+ 9045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:09:15.572066 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:09:15.574002 (+ 1936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:09:15.574765 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:09:15.574874 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:09:15.580718 (+ 5844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:09:15.580736 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:09:15.580755 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:09:15.580803 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:09:15.584901 (+ 4098us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:09:15.585950 (+ 1049us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:09:15.585962 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:09:15.585966 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:09:15.586153 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:09:15.586692 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:09:15.586698 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:09:15.586701 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:09:15.587241 (+ 540us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:09:15.587584 (+ 343us) server_negotiation.cc:1035] Waiting for connection context
0811 09:09:15.588196 (+ 612us) server_negotiation.cc:299] Negotiation successful
0811 09:09:15.588467 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260811 09:09:15.589646 16140 catalog_manager.cc:4841] TS c22db3988928400f86c90781e7e33d12 (127.28.154.130:43621): tablet aa2294f93df5402eb9873c5e14f801ed (table test-table [id=662fa245f3b74fdb9e4da1e0c9841567]) successfully deleted
I20260811 09:09:15.590615 16525 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:09:15.593170 17135 ts_tablet_manager.cc:1918] T aa2294f93df5402eb9873c5e14f801ed P 584ed0821f4c4d078dd295a60ebc56c8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:09:15.593371 17135 log.cc:1196] T aa2294f93df5402eb9873c5e14f801ed P 584ed0821f4c4d078dd295a60ebc56c8: Deleting WAL directory at /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439250192320-29290-0/minicluster-data/ts-2/wal/wals/aa2294f93df5402eb9873c5e14f801ed
I20260811 09:09:15.594051 17135 ts_tablet_manager.cc:1939] T aa2294f93df5402eb9873c5e14f801ed P 584ed0821f4c4d078dd295a60ebc56c8: Deleting consensus metadata
I20260811 09:09:15.596371 16136 catalog_manager.cc:4841] TS 584ed0821f4c4d078dd295a60ebc56c8 (127.28.154.131:37375): tablet aa2294f93df5402eb9873c5e14f801ed (table test-table [id=662fa245f3b74fdb9e4da1e0c9841567]) successfully deleted
I20260811 09:09:15.729760 17134 ts_tablet_manager.cc:1918] T aa2294f93df5402eb9873c5e14f801ed P d7a6ebbeb61841ad933cf189b99e9b92: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:09:15.730008 17134 log.cc:1196] T aa2294f93df5402eb9873c5e14f801ed P d7a6ebbeb61841ad933cf189b99e9b92: Deleting WAL directory at /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786439250192320-29290-0/minicluster-data/ts-0/wal/wals/aa2294f93df5402eb9873c5e14f801ed
I20260811 09:09:15.730445 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 16278
I20260811 09:09:15.759279 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 16540
W20260811 09:09:15.759223 16141 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.28.154.129:38953 (error 108)
I20260811 09:09:15.761508 17147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:09:15.760664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.129:38953 (local address 127.0.0.1:35480)
0811 09:09:15.761136 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0811 09:09:15.761288 (+ 152us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.129:38953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":222,"threads_started":1}
W20260811 09:09:15.761821 16141 catalog_manager.cc:4584] TS d7a6ebbeb61841ad933cf189b99e9b92 (127.28.154.129:38953): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet aa2294f93df5402eb9873c5e14f801ed: Network error: Client connection negotiation failed: client connection to 127.28.154.129:38953: connect: Connection refused (error 111)
Aug 11 09:09:15 dist-test-slave-xt4k krb5kdc[16082](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439352, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.154.130@KRBTEST.COM
I20260811 09:09:15.777791 17077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:09:15.760373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38506 (local address 127.28.154.130:43621)
0811 09:09:15.760539 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0811 09:09:15.760545 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:09:15.760566 (+ 21us) server_negotiation.cc:407] Connection header received
0811 09:09:15.760634 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:09:15.760639 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:09:15.760731 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:09:15.760840 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:09:15.762415 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:09:15.763619 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:09:15.765214 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:09:15.765898 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:09:15.766049 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:09:15.770601 (+ 4552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:09:15.770616 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:09:15.770633 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:09:15.770693 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:09:15.774782 (+ 4089us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:09:15.775681 (+ 899us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:09:15.775689 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:09:15.775691 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:09:15.775843 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:09:15.776261 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:09:15.776265 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:09:15.776268 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:09:15.776678 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:09:15.776929 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0811 09:09:15.777299 (+ 370us) server_negotiation.cc:299] Negotiation successful
0811 09:09:15.777475 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 09:09:15.778613 16765 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:09:15.815472 17147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:09:15.814948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.129:38953 (local address 127.0.0.1:35486)
0811 09:09:15.815175 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0811 09:09:15.815297 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.129:38953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260811 09:09:15.816728 17147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:09:15.816233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.129:38953 (local address 127.0.0.1:35500)
0811 09:09:15.816459 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0811 09:09:15.816553 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.129:38953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260811 09:09:15.877552 17147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:09:15.876988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.129:38953 (local address 127.0.0.1:35514)
0811 09:09:15.877248 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0811 09:09:15.877369 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.129:38953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260811 09:09:15.878676 17147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:09:15.878260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.129:38953 (local address 127.0.0.1:35530)
0811 09:09:15.878416 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0811 09:09:15.878504 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.129:38953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260811 09:09:15.895473 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 16540
I20260811 09:09:15.917538 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 16788
Aug 11 09:09:15 dist-test-slave-xt4k krb5kdc[16082](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439352, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.154.131@KRBTEST.COM
I20260811 09:09:15.933612 17079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:09:15.918668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56022 (local address 127.28.154.131:37375)
0811 09:09:15.918831 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 09:09:15.918836 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:09:15.918854 (+ 18us) server_negotiation.cc:407] Connection header received
0811 09:09:15.919007 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:09:15.919011 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:09:15.919080 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:09:15.919181 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:09:15.920503 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:09:15.921257 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:09:15.922591 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:09:15.923025 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:09:15.923132 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:09:15.927502 (+ 4370us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:09:15.927518 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:09:15.927538 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:09:15.927576 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:09:15.930424 (+ 2848us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:09:15.931176 (+ 752us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:09:15.931185 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:09:15.931194 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:09:15.931355 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:09:15.931743 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:09:15.931748 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:09:15.931750 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:09:15.932189 (+ 439us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:09:15.932565 (+ 376us) server_negotiation.cc:1035] Waiting for connection context
0811 09:09:15.933122 (+ 557us) server_negotiation.cc:299] Negotiation successful
0811 09:09:15.933331 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 09:09:15.934286 17029 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:09:15.959434 17147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:09:15.958832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.129:38953 (local address 127.0.0.1:35544)
0811 09:09:15.959125 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0811 09:09:15.959252 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.129:38953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101}
I20260811 09:09:15.960747 17147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:09:15.960225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.129:38953 (local address 127.0.0.1:35548)
0811 09:09:15.960477 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0811 09:09:15.960567 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.129:38953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137}
I20260811 09:09:16.024013 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 16788
I20260811 09:09:16.047488 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 16106
I20260811 09:09:16.048700 16223 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:09:16.162142 17147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:09:16.161638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.129:38953 (local address 127.0.0.1:35554)
0811 09:09:16.161856 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0811 09:09:16.161961 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.129:38953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260811 09:09:16.162313 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 16106
2026-08-11T09:09:16Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (4934 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:09:16 dist-test-slave-xt4k krb5kdc[17161](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:09:16 dist-test-slave-xt4k krb5kdc[17161](info): set up 2 sockets
Aug 11 09:09:16 dist-test-slave-xt4k krb5kdc[17161](info): commencing operation
krb5kdc: starting...
W20260811 09:09:17.470518 29290 mini_kdc.cc:121] Time spent starting KDC: real 1.258s user 0.000s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/krb5kdc/test-user.keytab.
Aug 11 09:09:17 dist-test-slave-xt4k krb5kdc[17161](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439357, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:09:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:09:17Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.28.154.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.28.154.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.28.154.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/krb5kdc/rangeradmin_127.28.154.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.28.154.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/krb5kdc/rangeradmin_127.28.154.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.28.154.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.28.154.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.28.154.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/krb5kdc/rangerlookup_127.28.154.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.28.154.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/krb5kdc/rangerlookup_127.28.154.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.28.154.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.28.154.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.28.154.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/krb5kdc/HTTP_127.28.154.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.28.154.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/krb5kdc/HTTP_127.28.154.148@KRBTEST.COM.keytab.
I20260811 09:09:17.903746 29290 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-tasklh2cYx/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/minicluster-data/postgres -l logfile start
2026-08-11 09:09:20.813 UTC [17223] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-11 09:09:20.813 UTC [17223] LOG: listening on IPv4 address "127.28.154.148", port 43895
2026-08-11 09:09:20.817 UTC [17223] LOG: listening on Unix socket "/tmp/.s.PGSQL.43895"
2026-08-11 09:09:20.838 UTC [17227] LOG: database system was shut down at 2026-08-11 09:09:19 UTC
2026-08-11 09:09:20.843 UTC [17223] LOG: database system is ready to accept connections
I20260811 09:09:22.005411 29290 mini_postgres.cc:96] Postgres bound to 43895
2026-08-11 09:09:22.031 UTC [17246] FATAL: database "slave" does not exist
127.28.154.148:43895 - accepting connections
I20260811 09:09:22.032123 29290 mini_ranger.cc:161] Starting Ranger...
I20260811 09:09:22.067890 29290 mini_ranger.cc:84] Created miniranger Postgres user
I20260811 09:09:22.610105 29290 mini_ranger.cc:87] Created ranger Postgres database
I20260811 09:09:22.610291 29290 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/minicluster-data/ranger-admin
2026-08-11 09:09:22,987 [I] DB FLAVOR :POSTGRES
2026-08-11 09:09:22,987 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 09:09:22,987 [I] Checking connection..
2026-08-11 09:09:22,987 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklh2cYx/build/asan/bin/postgresql.jar:/tmp/dist-test-tasklh2cYx/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.154.148:43895/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-11 09:09:23,265 [I] Checking connection passed.
2026-08-11 09:09:23,265 [I] --------- Verifying version history table ---------
2026-08-11 09:09:23,266 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklh2cYx/build/asan/bin/postgresql.jar:/tmp/dist-test-tasklh2cYx/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.154.148:43895/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 09:09:23,546 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 09:09:23,547 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklh2cYx/build/asan/bin/postgresql.jar:/tmp/dist-test-tasklh2cYx/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.154.148:43895/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 09:09:23,804 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 09:09:23,804 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-11 09:09:23,805 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklh2cYx/build/asan/bin/postgresql.jar:/tmp/dist-test-tasklh2cYx/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.154.148:43895/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasklh2cYx/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-11 09:09:24.044 UTC [17362] WARNING: there is no transaction in progress
2026-08-11 09:09:24,051 [I] create_dbversion_catalog.sql file imported successfully
2026-08-11 09:09:24,052 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklh2cYx/build/asan/bin/postgresql.jar:/tmp/dist-test-tasklh2cYx/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.154.148:43895/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 09:09:24,334 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-11 09:09:24,334 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 09:09:24,334 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklh2cYx/build/asan/bin/postgresql.jar:/tmp/dist-test-tasklh2cYx/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.154.148:43895/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-11 09:09:24,604 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklh2cYx/build/asan/bin/postgresql.jar:/tmp/dist-test-tasklh2cYx/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.154.148:43895/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-11 09:09:24,854 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklh2cYx/build/asan/bin/postgresql.jar:/tmp/dist-test-tasklh2cYx/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.154.148:43895/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-xt4k.c.gcp-upstream.internal','N') ;"
2026-08-11 09:09:25,124 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklh2cYx/build/asan/bin/postgresql.jar:/tmp/dist-test-tasklh2cYx/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.154.148:43895/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 09:09:25,411 [I] Table x_portal_user does not exist in database ranger
2026-08-11 09:09:25,411 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklh2cYx/build/asan/bin/postgresql.jar:/tmp/dist-test-tasklh2cYx/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.154.148:43895/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 09:09:25,669 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-11 09:09:25,669 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklh2cYx/build/asan/bin/postgresql.jar:/tmp/dist-test-tasklh2cYx/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.154.148:43895/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 09:09:25,936 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklh2cYx/build/asan/bin/postgresql.jar:/tmp/dist-test-tasklh2cYx/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.154.148:43895/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 09:09:26,199 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-11 09:09:26,199 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklh2cYx/build/asan/bin/postgresql.jar:/tmp/dist-test-tasklh2cYx/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.154.148:43895/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasklh2cYx/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-11 09:09:27.194 UTC [17566] WARNING: there is no transaction in progress
2026-08-11 09:09:27.211 UTC [17566] WARNING: there is no transaction in progress
2026-08-11 09:09:27.228 UTC [17566] WARNING: there is no transaction in progress
2026-08-11 09:09:27.246 UTC [17566] WARNING: there is no transaction in progress
2026-08-11 09:09:27.262 UTC [17566] WARNING: there is no transaction in progress
2026-08-11 09:09:27.280 UTC [17566] WARNING: there is no transaction in progress
2026-08-11 09:09:27.338 UTC [17566] WARNING: there is no transaction in progress
2026-08-11 09:09:27.346 UTC [17566] WARNING: there is no transaction in progress
2026-08-11 09:09:27.361 UTC [17566] WARNING: there is no transaction in progress
2026-08-11 09:09:27.373 UTC [17566] WARNING: there is no transaction in progress
2026-08-11 09:09:27.384 UTC [17566] WARNING: there is no transaction in progress
2026-08-11 09:09:27.399 UTC [17566] WARNING: there is no transaction in progress
2026-08-11 09:09:27.409 UTC [17566] WARNING: there is no transaction in progress
2026-08-11 09:09:27.416 UTC [17566] WARNING: there is no transaction in progress
2026-08-11 09:09:27.424 UTC [17566] WARNING: there is no transaction in progress
2026-08-11 09:09:27.431 UTC [17566] WARNING: there is no transaction in progress
2026-08-11 09:09:28,106 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-11 09:09:28,106 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklh2cYx/build/asan/bin/postgresql.jar:/tmp/dist-test-tasklh2cYx/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.154.148:43895/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-11 09:09:28,368 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-11 09:09:28,368 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklh2cYx/build/asan/bin/postgresql.jar:/tmp/dist-test-tasklh2cYx/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.154.148:43895/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 09:09:28,609 [I] Table x_portal_user already exists in database 'ranger'
2026-08-11 09:09:28,609 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklh2cYx/build/asan/bin/postgresql.jar:/tmp/dist-test-tasklh2cYx/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.154.148:43895/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 09:09:28,864 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-11 09:09:28,864 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklh2cYx/build/asan/bin/postgresql.jar:/tmp/dist-test-tasklh2cYx/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.154.148:43895/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 09:09:29,119 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklh2cYx/build/asan/bin/postgresql.jar:/tmp/dist-test-tasklh2cYx/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.154.148:43895/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 09:09:29,372 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklh2cYx/build/asan/bin/postgresql.jar:/tmp/dist-test-tasklh2cYx/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.154.148:43895/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-xt4k.c.gcp-upstream.internal';"
2026-08-11 09:09:29,611 [I] CORE_DB_SCHEMA import status has been updated
2026-08-11 09:09:29,612 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklh2cYx/build/asan/bin/postgresql.jar:/tmp/dist-test-tasklh2cYx/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.154.148:43895/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-11 09:09:29,867 [I] DB_PATCHES have already been applied
I20260811 09:09:29.872952 29290 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/minicluster-data/ranger-admin:/tmp/dist-test-tasklh2cYx/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-tasklh2cYx/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-tasklh2cYx/build/asan/bin/postgresql.jar:/tmp/dist-test-tasklh2cYx/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260811 09:09:29.873078 29290 mini_ranger.cc:193] Using host: 127.28.154.148
I20260811 09:09:29.894348 29290 mini_ranger.cc:227] Ranger admin URL: http://127.28.154.148:42961
Aug 11, 2026 9:09:30 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 9:09:30 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 9:09:30 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/minicluster-data/ranger-admin/ews/webapp
Aug 11, 2026 9:09:30 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 11, 2026 9:09:30 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 11, 2026 9:09:30 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/minicluster-data/ranger-admin/ews/webapp].
Aug 11, 2026 9:09:30 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260811 09:09:31.261899 29290 mini_ranger.cc:160] Time spent starting Ranger: real 9.230s user 0.002s sys 0.067s
Aug 11, 2026 9:09:52 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 11, 2026 9:09:52 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 11, 2026 9:09:52 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 11, 2026 9:09:52 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 11, 2026 9:09:52 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 11, 2026 9:09:52 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 11, 2026 9:09:52 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 11, 2026 9:09:52 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 11, 2026 9:09:52 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 11, 2026 9:09:52 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 11, 2026 9:09:52 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 11, 2026 9:09:52 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 11, 2026 9:09:52 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 11, 2026 9:09:52 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 11, 2026 9:09:52 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 11, 2026 9:09:52 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 11, 2026 9:09:52 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 11, 2026 9:09:53 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260811 09:09:53.961855 29290 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.28.154.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.28.154.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.28.154.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/krb5kdc/rangerkms_127.28.154.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.28.154.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/krb5kdc/rangerkms_127.28.154.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.28.154.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.28.154.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.28.154.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/krb5kdc/HTTP_127.28.154.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.28.154.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/krb5kdc/HTTP_127.28.154.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 11 09:09:54 dist-test-slave-xt4k krb5kdc[17161](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439394, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260811 09:09:54.211104 29290 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260811 09:09:54.246040 29290 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260811 09:09:55.239567 29290 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260811 09:09:55.239712 29290 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/minicluster-data/ranger-kms
I20260811 09:09:55.239866 29290 mini_ranger_kms.cc:227] Using postgres at 127.28.154.148:43895
2026-08-11 09:09:55,835 [I] DB FLAVOR :POSTGRES
2026-08-11 09:09:55,836 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 09:09:55,836 [I] Checking connection
2026-08-11 09:09:55,836 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklh2cYx/build/asan/bin/postgresql.jar:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.154.148:43895/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-11 09:09:56,095 [I] connection success
2026-08-11 09:09:56,095 [I] --------- Verifying Ranger DB tables ---------
2026-08-11 09:09:56,095 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-11 09:09:56,095 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklh2cYx/build/asan/bin/postgresql.jar:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.154.148:43895/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-11 09:09:56,345 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-11 09:09:56,345 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 09:09:56,345 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-11 09:09:56,345 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklh2cYx/build/asan/bin/postgresql.jar:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.154.148:43895/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-11 09:09:56,625 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260811 09:09:57.162750 29290 mini_ranger_kms.cc:310] Created kms service
I20260811 09:09:57.385118 29290 mini_ranger_kms.cc:326] Created kudu user
I20260811 09:09:57.437258 29290 mini_ranger_kms.cc:343] Created rangerkms user
Aug 11 09:09:57 dist-test-slave-xt4k krb5kdc[17161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439394, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.28.154.148@KRBTEST.COM
I20260811 09:09:57.904907 29290 mini_ranger_kms.cc:384] Added ranger policy
I20260811 09:09:57.906390 29290 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/minicluster-data/ranger-kms:/tmp/dist-test-tasklh2cYx/build/asan/bin/postgresql.jar:/tmp/dist-test-tasklh2cYx/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-tasklh2cYx/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260811 09:09:57.906467 29290 mini_ranger_kms.cc:240] Using host: 127.28.154.148
I20260811 09:09:57.933828 29290 mini_ranger_kms.cc:276] Ranger KMS PID: 17894
I20260811 09:09:57.934036 29290 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.28.154.148:33021
Aug 11, 2026 9:09:58 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 9:09:58 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 9:09:59 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 11, 2026 9:09:59 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 11, 2026 9:09:59 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/minicluster-data/ranger-kms/ews/webapp].
Aug 11, 2026 9:09:59 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 11 09:10:00 dist-test-slave-xt4k krb5kdc[17161](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786439400, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.28.154.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:10:00.739550 29290 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.528s user 0.125s sys 0.526s
I20260811 09:10:00.739945 29290 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260811 09:10:00.740183 29290 mini_ranger_kms.cc:401] 127.28.154.148:33021/kms/v1/keys
Aug 11 09:10:00 dist-test-slave-xt4k krb5kdc[17161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439400, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.28.154.148@KRBTEST.COM for HTTP/127.28.154.148@KRBTEST.COM
Aug 11 09:10:01 dist-test-slave-xt4k krb5kdc[17161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439400, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.28.154.148@KRBTEST.COM for HTTP/127.28.154.148@KRBTEST.COM
Aug 11 09:10:03 dist-test-slave-xt4k krb5kdc[17161](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439403, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.28.154.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.154.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.154.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab.
Entry for principal kudu/127.28.154.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab.
I20260811 09:10:03.626878 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.154.190:38573
--webserver_interface=127.28.154.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab
--principal=kudu/127.28.154.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:41083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.154.190:38573
--ranger_config_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.28.154.148:33021/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:10:04.026515 17992 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:10:04.026916 17992 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:10:04.027061 17992 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:10:04.035415 17992 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:10:04.035563 17992 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:10:04.035657 17992 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:10:04.035766 17992 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:10:04.035837 17992 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:10:04.046926 17992 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:41083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/minicluster-data/master-0/wal
--ranger_kms_url=127.28.154.148:33021/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.28.154.190:38573
--ranger_config_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab
--principal=kudu/127.28.154.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.154.190:38573
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.154.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:10:04.048301 17992 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:10:04.049685 17992 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:10:04.065109 18002 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:10:04.065289 18001 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:10:04.065207 18006 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:10:04.066429 17992 server_base.cc:1034] running on GCE node
I20260811 09:10:04.067378 17992 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:10:04.069308 17992 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:10:04.070545 17992 hybrid_clock.cc:648] HybridClock initialized: now 1786439404070502 us; error 52 us; skew 500 ppm
I20260811 09:10:04.071002 17992 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:10:04 dist-test-slave-xt4k krb5kdc[17161](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439404, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:10:04.078963 17992 init.cc:377] Logged in from keytab as kudu/127.28.154.190@KRBTEST.COM (short username kudu)
I20260811 09:10:04.081441 17992 webserver.cc:460] Webserver started at http://127.28.154.190:46487/ using document root <none> and password file <none>
I20260811 09:10:04.082372 17992 fs_manager.cc:359] Metadata directory not provided
I20260811 09:10:04.082600 17992 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:10:04.083088 17992 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 09:10:04 dist-test-slave-xt4k krb5kdc[17161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439404, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.190@KRBTEST.COM for HTTP/127.28.154.148@KRBTEST.COM
I20260811 09:10:04.154067 17992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/minicluster-data/master-0/data/instance:
uuid: "6da0e5689c6d485387173849aa7f4ff4"
format_stamp: "Formatted at 2026-08-11 09:10:04 on dist-test-slave-xt4k"
server_key: "da711e62388efed8c5ee9ebcfa475fbd"
server_key_iv: "942de26e3aefab26f5441aa50be2abe0"
server_key_version: "kuduclusterkey@0"
I20260811 09:10:04.155086 17992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/minicluster-data/master-0/wal/instance:
uuid: "6da0e5689c6d485387173849aa7f4ff4"
format_stamp: "Formatted at 2026-08-11 09:10:04 on dist-test-slave-xt4k"
server_key: "da711e62388efed8c5ee9ebcfa475fbd"
server_key_iv: "942de26e3aefab26f5441aa50be2abe0"
server_key_version: "kuduclusterkey@0"
I20260811 09:10:04.162771 17992 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.000s sys 0.008s
I20260811 09:10:04.186062 18016 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:10:04.187999 17992 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 09:10:04.188310 17992 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/minicluster-data/master-0/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/minicluster-data/master-0/wal
uuid: "6da0e5689c6d485387173849aa7f4ff4"
format_stamp: "Formatted at 2026-08-11 09:10:04 on dist-test-slave-xt4k"
server_key: "da711e62388efed8c5ee9ebcfa475fbd"
server_key_iv: "942de26e3aefab26f5441aa50be2abe0"
server_key_version: "kuduclusterkey@0"
I20260811 09:10:04.188633 17992 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:10:04.218407 17992 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:10:04.224031 17992 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:10:04.224387 17992 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:10:04.270859 17992 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.190:38573
I20260811 09:10:04.271046 18099 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.190:38573 every 8 connection(s)
I20260811 09:10:04.272817 17992 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
I20260811 09:10:04.276827 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 17992
I20260811 09:10:04.277019 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/minicluster-data/master-0/wal/instance
I20260811 09:10:04.277868 29290 external_mini_cluster.cc:1428] Setting key f05b344812a4d4f2efc4b496d06d7597
I20260811 09:10:04.278831 18100 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 09:10:04 dist-test-slave-xt4k krb5kdc[17161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439403, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.154.190@KRBTEST.COM
I20260811 09:10:04.301916 18100 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6da0e5689c6d485387173849aa7f4ff4: Bootstrap starting.
I20260811 09:10:04.308324 18100 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6da0e5689c6d485387173849aa7f4ff4: Neither blocks nor log segments found. Creating new log.
I20260811 09:10:04.310348 18100 log.cc:824] T 00000000000000000000000000000000 P 6da0e5689c6d485387173849aa7f4ff4: Log is configured to *not* fsync() on all Append() calls
I20260811 09:10:04.312168 18106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:10:04.281300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33798 (local address 127.28.154.190:38573)
0811 09:10:04.283007 (+ 1707us) server_negotiation.cc:206] Beginning negotiation
0811 09:10:04.283025 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 09:10:04.283116 (+ 91us) server_negotiation.cc:407] Connection header received
0811 09:10:04.286335 (+ 3219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:10:04.286374 (+ 39us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:10:04.287065 (+ 691us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:10:04.287673 (+ 608us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:10:04.289368 (+ 1695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:10:04.292139 (+ 2771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:10:04.293715 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:10:04.295011 (+ 1296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:10:04.295268 (+ 257us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:10:04.300154 (+ 4886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:10:04.300170 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:10:04.300195 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:10:04.300248 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:10:04.306420 (+ 6172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:10:04.307488 (+ 1068us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:10:04.307498 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:10:04.307516 (+ 18us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:10:04.307772 (+ 256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:10:04.308382 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:10:04.308387 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:10:04.308390 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:10:04.309369 (+ 979us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:10:04.309828 (+ 459us) server_negotiation.cc:1035] Waiting for connection context
0811 09:10:04.310188 (+ 360us) server_negotiation.cc:299] Negotiation successful
0811 09:10:04.310654 (+ 466us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1380,"thread_start_us":862,"threads_started":1}
I20260811 09:10:04.315075 18100 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6da0e5689c6d485387173849aa7f4ff4: No bootstrap required, opened a new log
I20260811 09:10:04.328697 18100 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6da0e5689c6d485387173849aa7f4ff4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6da0e5689c6d485387173849aa7f4ff4" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 38573 } }
I20260811 09:10:04.329406 18100 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6da0e5689c6d485387173849aa7f4ff4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:10:04.329592 18100 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6da0e5689c6d485387173849aa7f4ff4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6da0e5689c6d485387173849aa7f4ff4, State: Initialized, Role: FOLLOWER
I20260811 09:10:04.330447 18100 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6da0e5689c6d485387173849aa7f4ff4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6da0e5689c6d485387173849aa7f4ff4" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 38573 } }
I20260811 09:10:04.330885 18100 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6da0e5689c6d485387173849aa7f4ff4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:10:04.331151 18100 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6da0e5689c6d485387173849aa7f4ff4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:10:04.331491 18100 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6da0e5689c6d485387173849aa7f4ff4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:10:04.335063 18100 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6da0e5689c6d485387173849aa7f4ff4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6da0e5689c6d485387173849aa7f4ff4" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 38573 } }
I20260811 09:10:04.335669 18100 leader_election.cc:302] T 00000000000000000000000000000000 P 6da0e5689c6d485387173849aa7f4ff4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6da0e5689c6d485387173849aa7f4ff4; no voters:
I20260811 09:10:04.336562 18100 leader_election.cc:288] T 00000000000000000000000000000000 P 6da0e5689c6d485387173849aa7f4ff4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:10:04.336907 18108 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6da0e5689c6d485387173849aa7f4ff4 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:10:04.338474 18108 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6da0e5689c6d485387173849aa7f4ff4 [term 1 LEADER]: Becoming Leader. State: Replica: 6da0e5689c6d485387173849aa7f4ff4, State: Running, Role: LEADER
I20260811 09:10:04.339366 18108 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6da0e5689c6d485387173849aa7f4ff4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6da0e5689c6d485387173849aa7f4ff4" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 38573 } }
I20260811 09:10:04.339722 18100 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6da0e5689c6d485387173849aa7f4ff4 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:10:04.344460 18109 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6da0e5689c6d485387173849aa7f4ff4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6da0e5689c6d485387173849aa7f4ff4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6da0e5689c6d485387173849aa7f4ff4" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 38573 } } }
I20260811 09:10:04.344594 18113 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6da0e5689c6d485387173849aa7f4ff4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6da0e5689c6d485387173849aa7f4ff4. Latest consensus state: current_term: 1 leader_uuid: "6da0e5689c6d485387173849aa7f4ff4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6da0e5689c6d485387173849aa7f4ff4" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 38573 } } }
I20260811 09:10:04.345498 18113 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6da0e5689c6d485387173849aa7f4ff4 [sys.catalog]: This master's current role is: LEADER
I20260811 09:10:04.345506 18109 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6da0e5689c6d485387173849aa7f4ff4 [sys.catalog]: This master's current role is: LEADER
I20260811 09:10:04.353781 18125 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:10:04.363620 18125 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:10:04.364854 18100 ranger_client.cc:318] Using new properties file: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 09:10:04.402411 18125 catalog_manager.cc:1347] Generated new cluster ID: 635c4ba1615d42e58d9155aecfbe40be
I20260811 09:10:04.402673 18125 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:10:04.433790 18125 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:10:04.436604 18125 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:10:04.455405 18125 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6da0e5689c6d485387173849aa7f4ff4: Generated new TSK 0
I20260811 09:10:04.456910 18125 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.28.154.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.154.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.154.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.129.keytab.
Entry for principal kudu/127.28.154.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.129.keytab.
I20260811 09:10:05.317742 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.28.154.148:33021/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.154.129:0
--local_ip_for_outbound_sockets=127.28.154.129
--webserver_interface=127.28.154.129
--webserver_port=0
--tserver_master_addrs=127.28.154.190:38573
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.129.keytab
--principal=kudu/127.28.154.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:41083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 09:10:05 dist-test-slave-xt4k krb5kdc[17161](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786439405, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260811 09:10:05.980012 18190 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:10:05.980425 18190 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:10:05.980621 18190 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:10:05.996838 18190 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:10:05.997181 18190 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:10:05.997555 18190 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.154.129
I20260811 09:10:06.018497 18190 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:41083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.28.154.148:33021/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.129.keytab
--principal=kudu/127.28.154.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.154.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.154.129
--webserver_port=0
--tserver_master_addrs=127.28.154.190:38573
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.154.129
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:10:06.020843 18190 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:10:06.023124 18190 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:10:06.045794 18209 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:10:06.047169 18214 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:10:06.047369 18211 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:10:06.048698 18190 server_base.cc:1034] running on GCE node
I20260811 09:10:06.049469 18190 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:10:06.050791 18190 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:10:06.052017 18190 hybrid_clock.cc:648] HybridClock initialized: now 1786439406051942 us; error 72 us; skew 500 ppm
I20260811 09:10:06.052440 18190 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:10:06 dist-test-slave-xt4k krb5kdc[17161](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439406, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:10:06.057724 18190 init.cc:377] Logged in from keytab as kudu/127.28.154.129@KRBTEST.COM (short username kudu)
I20260811 09:10:06.060480 18190 webserver.cc:460] Webserver started at http://127.28.154.129:41719/ using document root <none> and password file <none>
I20260811 09:10:06.061712 18190 fs_manager.cc:359] Metadata directory not provided
I20260811 09:10:06.062054 18190 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:10:06.062729 18190 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 09:10:06 dist-test-slave-xt4k krb5kdc[17161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439406, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.129@KRBTEST.COM for HTTP/127.28.154.148@KRBTEST.COM
I20260811 09:10:06.085001 18190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/minicluster-data/ts-0/data/instance:
uuid: "55110c3cf110404a8a7fa2591be4411f"
format_stamp: "Formatted at 2026-08-11 09:10:06 on dist-test-slave-xt4k"
server_key: "eebeb4697f12952ad7ef5468498f6c18"
server_key_iv: "514eeb00c5c30c8addace08219ebacc0"
server_key_version: "kuduclusterkey@0"
I20260811 09:10:06.086804 18190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/minicluster-data/ts-0/wal/instance:
uuid: "55110c3cf110404a8a7fa2591be4411f"
format_stamp: "Formatted at 2026-08-11 09:10:06 on dist-test-slave-xt4k"
server_key: "eebeb4697f12952ad7ef5468498f6c18"
server_key_iv: "514eeb00c5c30c8addace08219ebacc0"
server_key_version: "kuduclusterkey@0"
I20260811 09:10:06.107740 18190 fs_manager.cc:691] Time spent creating directory manager: real 0.020s user 0.007s sys 0.011s
I20260811 09:10:06.149019 18225 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:10:06.151227 18190 fs_manager.cc:725] Time spent opening block manager: real 0.022s user 0.000s sys 0.018s
I20260811 09:10:06.151521 18190 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/minicluster-data/ts-0/wal
uuid: "55110c3cf110404a8a7fa2591be4411f"
format_stamp: "Formatted at 2026-08-11 09:10:06 on dist-test-slave-xt4k"
server_key: "eebeb4697f12952ad7ef5468498f6c18"
server_key_iv: "514eeb00c5c30c8addace08219ebacc0"
server_key_version: "kuduclusterkey@0"
I20260811 09:10:06.151866 18190 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:10:06.192668 18190 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:10:06.200503 18190 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:10:06.201231 18190 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:10:06.209929 18190 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:10:06.216086 18190 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:10:06.216265 18190 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:10:06.216619 18190 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:10:06.216779 18190 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 09:10:06 dist-test-slave-xt4k krb5kdc[17161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439405, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.190@KRBTEST.COM for HTTP/127.28.154.148@KRBTEST.COM
I20260811 09:10:06.521632 18190 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.129:39433
I20260811 09:10:06.521651 18431 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.129:39433 every 8 connection(s)
I20260811 09:10:06.524546 18190 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/minicluster-data/ts-0/data/info.pb
I20260811 09:10:06.529453 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 18190
I20260811 09:10:06.529697 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/minicluster-data/ts-0/wal/instance
I20260811 09:10:06.531102 29290 external_mini_cluster.cc:1428] Setting key c4949e435538bf00fdc57e4263a54632
Aug 11 09:10:06 dist-test-slave-xt4k krb5kdc[17161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439406, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.129@KRBTEST.COM for kudu/127.28.154.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.28.154.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.154.130@KRBTEST.COM" created.
I20260811 09:10:06.582166 18440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:10:06.529000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:38573 (local address 127.28.154.129:55641)
0811 09:10:06.542583 (+ 13583us) negotiation.cc:107] Waiting for socket to connect
0811 09:10:06.542649 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0811 09:10:06.547952 (+ 5303us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:10:06.553994 (+ 6042us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:10:06.554009 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:10:06.555132 (+ 1123us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:10:06.556632 (+ 1500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:10:06.556665 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:10:06.560287 (+ 3622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:10:06.560293 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:10:06.562887 (+ 2594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:10:06.562929 (+ 42us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:10:06.564929 (+ 2000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:10:06.564938 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:10:06.565516 (+ 578us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:10:06.566480 (+ 964us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:10:06.566512 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:10:06.570226 (+ 3714us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:10:06.576164 (+ 5938us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:10:06.576177 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:10:06.576200 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:10:06.576982 (+ 782us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:10:06.577621 (+ 639us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:10:06.577627 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:10:06.577632 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:10:06.578006 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:10:06.578988 (+ 982us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:10:06.578997 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:10:06.579586 (+ 589us) client_negotiation.cc:769] Sending connection context
0811 09:10:06.579921 (+ 335us) client_negotiation.cc:240] Negotiation successful
0811 09:10:06.580616 (+ 695us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13128,"thread_start_us":16541,"threads_started":1}
I20260811 09:10:06.589740 18436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:10:06.528208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.129:55641 (local address 127.28.154.190:38573)
0811 09:10:06.553243 (+ 25035us) server_negotiation.cc:206] Beginning negotiation
0811 09:10:06.553250 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:10:06.553278 (+ 28us) server_negotiation.cc:407] Connection header received
0811 09:10:06.553350 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:10:06.553357 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:10:06.553447 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:10:06.556862 (+ 3415us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:10:06.557212 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:10:06.558755 (+ 1543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:10:06.563167 (+ 4412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:10:06.564167 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:10:06.564303 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:10:06.570625 (+ 6322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:10:06.570634 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:10:06.570639 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:10:06.570714 (+ 75us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:10:06.575901 (+ 5187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:10:06.577170 (+ 1269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:10:06.577175 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:10:06.577179 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:10:06.577466 (+ 287us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:10:06.578178 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:10:06.578184 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:10:06.578188 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:10:06.578731 (+ 543us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:10:06.578901 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0811 09:10:06.588754 (+ 9853us) server_negotiation.cc:299] Negotiation successful
0811 09:10:06.589067 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24855,"thread_start_us":28155,"threads_started":1}
I20260811 09:10:06.592388 18432 heartbeater.cc:347] Connected to a master server at 127.28.154.190:38573
I20260811 09:10:06.592826 18432 heartbeater.cc:464] Registering TS with master...
I20260811 09:10:06.593863 18432 heartbeater.cc:511] Master 127.28.154.190:38573 requested a full tablet report, sending...
I20260811 09:10:06.597127 18041 ts_manager.cc:194] Registered new tserver with Master: 55110c3cf110404a8a7fa2591be4411f (127.28.154.129:39433)
I20260811 09:10:06.599383 18041 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.154.129@KRBTEST.COM'} at 127.28.154.129:55641
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.154.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.130.keytab.
Entry for principal kudu/127.28.154.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.130.keytab.
I20260811 09:10:06.620090 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.28.154.148:33021/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.154.130:0
--local_ip_for_outbound_sockets=127.28.154.130
--webserver_interface=127.28.154.130
--webserver_port=0
--tserver_master_addrs=127.28.154.190:38573
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.130.keytab
--principal=kudu/127.28.154.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:41083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 09:10:06 dist-test-slave-xt4k krb5kdc[17161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439405, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.190@KRBTEST.COM for HTTP/127.28.154.148@KRBTEST.COM
W20260811 09:10:07.414316 18446 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:10:07.414831 18446 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:10:07.415094 18446 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:10:07.425825 18446 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:10:07.426028 18446 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:10:07.426286 18446 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.154.130
I20260811 09:10:07.440904 18446 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:41083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.28.154.148:33021/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.130.keytab
--principal=kudu/127.28.154.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.154.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.154.130
--webserver_port=0
--tserver_master_addrs=127.28.154.190:38573
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.154.130
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:10:07.442618 18446 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:10:07.444110 18446 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:10:07.462889 18473 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:10:07.463977 18478 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:10:07.464298 18475 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:10:07.465942 18446 server_base.cc:1034] running on GCE node
I20260811 09:10:07.466732 18446 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:10:07.468050 18446 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:10:07.469343 18446 hybrid_clock.cc:648] HybridClock initialized: now 1786439407469217 us; error 44 us; skew 500 ppm
I20260811 09:10:07.469836 18446 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:10:07 dist-test-slave-xt4k krb5kdc[17161](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439407, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:10:07.475225 18446 init.cc:377] Logged in from keytab as kudu/127.28.154.130@KRBTEST.COM (short username kudu)
I20260811 09:10:07.477989 18446 webserver.cc:460] Webserver started at http://127.28.154.130:37141/ using document root <none> and password file <none>
I20260811 09:10:07.479054 18446 fs_manager.cc:359] Metadata directory not provided
I20260811 09:10:07.479276 18446 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:10:07.479770 18446 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 09:10:07 dist-test-slave-xt4k krb5kdc[17161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439407, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.130@KRBTEST.COM for HTTP/127.28.154.148@KRBTEST.COM
I20260811 09:10:07.497749 18446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/minicluster-data/ts-1/data/instance:
uuid: "f3c7f757dc6448f98f5106989efea337"
format_stamp: "Formatted at 2026-08-11 09:10:07 on dist-test-slave-xt4k"
server_key: "0b1653d070c39b22f8422022bb435732"
server_key_iv: "eb856e5d9cf39cb6f25b62ff3809158e"
server_key_version: "kuduclusterkey@0"
I20260811 09:10:07.498920 18446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/minicluster-data/ts-1/wal/instance:
uuid: "f3c7f757dc6448f98f5106989efea337"
format_stamp: "Formatted at 2026-08-11 09:10:07 on dist-test-slave-xt4k"
server_key: "0b1653d070c39b22f8422022bb435732"
server_key_iv: "eb856e5d9cf39cb6f25b62ff3809158e"
server_key_version: "kuduclusterkey@0"
I20260811 09:10:07.507231 18446 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.004s sys 0.004s
I20260811 09:10:07.527022 18487 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:10:07.528668 18446 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260811 09:10:07.528877 18446 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/minicluster-data/ts-1/wal
uuid: "f3c7f757dc6448f98f5106989efea337"
format_stamp: "Formatted at 2026-08-11 09:10:07 on dist-test-slave-xt4k"
server_key: "0b1653d070c39b22f8422022bb435732"
server_key_iv: "eb856e5d9cf39cb6f25b62ff3809158e"
server_key_version: "kuduclusterkey@0"
I20260811 09:10:07.529126 18446 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:10:07.546509 18446 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:10:07.554306 18446 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:10:07.554942 18446 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:10:07.556869 18446 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:10:07.560094 18446 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:10:07.560267 18446 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:10:07.560551 18446 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:10:07.560698 18446 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:10:07.603684 18432 heartbeater.cc:503] Master 127.28.154.190:38573 was elected leader, sending a full tablet report...
I20260811 09:10:07.667115 18446 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.130:35151
I20260811 09:10:07.667148 18684 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.130:35151 every 8 connection(s)
I20260811 09:10:07.669409 18446 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/minicluster-data/ts-1/data/info.pb
I20260811 09:10:07.674669 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 18446
I20260811 09:10:07.674877 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/minicluster-data/ts-1/wal/instance
I20260811 09:10:07.675510 29290 external_mini_cluster.cc:1428] Setting key 213c79fa5ae9b108d2680a0891697d18
Aug 11 09:10:07 dist-test-slave-xt4k krb5kdc[17161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439407, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.130@KRBTEST.COM for kudu/127.28.154.190@KRBTEST.COM
I20260811 09:10:07.703663 18690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:10:07.673482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.130:51953 (local address 127.28.154.190:38573)
0811 09:10:07.674144 (+ 662us) server_negotiation.cc:206] Beginning negotiation
0811 09:10:07.674151 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:10:07.678547 (+ 4396us) server_negotiation.cc:407] Connection header received
0811 09:10:07.681071 (+ 2524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:10:07.681077 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:10:07.681173 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:10:07.681299 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:10:07.683975 (+ 2676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:10:07.685647 (+ 1672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:10:07.688936 (+ 3289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:10:07.689614 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:10:07.689749 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:10:07.694853 (+ 5104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:10:07.694860 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:10:07.694865 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:10:07.694962 (+ 97us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:10:07.699545 (+ 4583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:10:07.700563 (+ 1018us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:10:07.700570 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:10:07.700573 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:10:07.700792 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:10:07.701426 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:10:07.701430 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:10:07.701432 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:10:07.701789 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:10:07.701940 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0811 09:10:07.702821 (+ 881us) server_negotiation.cc:299] Negotiation successful
0811 09:10:07.703075 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":485,"thread_start_us":327,"threads_started":1}
I20260811 09:10:07.704952 18692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:10:07.674321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:38573 (local address 127.28.154.130:51953)
0811 09:10:07.677252 (+ 2931us) negotiation.cc:107] Waiting for socket to connect
0811 09:10:07.677308 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 09:10:07.680705 (+ 3397us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:10:07.681649 (+ 944us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:10:07.681663 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:10:07.682515 (+ 852us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:10:07.683746 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:10:07.683777 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:10:07.685866 (+ 2089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:10:07.685872 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:10:07.688728 (+ 2856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:10:07.688754 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:10:07.690011 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:10:07.690021 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:10:07.690416 (+ 395us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:10:07.691348 (+ 932us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:10:07.691425 (+ 77us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:10:07.694571 (+ 3146us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:10:07.699744 (+ 5173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:10:07.699756 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:10:07.699774 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:10:07.700417 (+ 643us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:10:07.700920 (+ 503us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:10:07.700926 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:10:07.700930 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:10:07.701294 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:10:07.701928 (+ 634us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:10:07.701937 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:10:07.702354 (+ 417us) client_negotiation.cc:769] Sending connection context
0811 09:10:07.702673 (+ 319us) client_negotiation.cc:240] Negotiation successful
0811 09:10:07.703392 (+ 719us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2525,"thread_start_us":2671,"threads_started":1}
I20260811 09:10:07.706682 18687 heartbeater.cc:347] Connected to a master server at 127.28.154.190:38573
I20260811 09:10:07.707055 18687 heartbeater.cc:464] Registering TS with master...
I20260811 09:10:07.708218 18687 heartbeater.cc:511] Master 127.28.154.190:38573 requested a full tablet report, sending...
I20260811 09:10:07.710145 18041 ts_manager.cc:194] Registered new tserver with Master: f3c7f757dc6448f98f5106989efea337 (127.28.154.130:35151)
I20260811 09:10:07.711117 18041 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.154.130@KRBTEST.COM'} at 127.28.154.130:51953
WARNING: no policy specified for kudu/127.28.154.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.154.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.154.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.131.keytab.
Entry for principal kudu/127.28.154.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.131.keytab.
I20260811 09:10:07.759672 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.28.154.148:33021/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.154.131:0
--local_ip_for_outbound_sockets=127.28.154.131
--webserver_interface=127.28.154.131
--webserver_port=0
--tserver_master_addrs=127.28.154.190:38573
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.131.keytab
--principal=kudu/127.28.154.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:41083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:10:08.443349 18700 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:10:08.443761 18700 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:10:08.443993 18700 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:10:08.455664 18700 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:10:08.455957 18700 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:10:08.456234 18700 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.154.131
I20260811 09:10:08.472400 18700 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:41083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.28.154.148:33021/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.131.keytab
--principal=kudu/127.28.154.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.154.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.154.131
--webserver_port=0
--tserver_master_addrs=127.28.154.190:38573
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.154.131
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:10:08.474150 18700 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:10:08.475548 18700 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:10:08.494822 18708 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:10:08.496106 18711 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:10:08.496165 18709 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:10:08.497094 18700 server_base.cc:1034] running on GCE node
I20260811 09:10:08.497886 18700 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:10:08.499614 18700 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:10:08.500870 18700 hybrid_clock.cc:648] HybridClock initialized: now 1786439408500732 us; error 93 us; skew 500 ppm
I20260811 09:10:08.501349 18700 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:10:08 dist-test-slave-xt4k krb5kdc[17161](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439408, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:10:08.506079 18700 init.cc:377] Logged in from keytab as kudu/127.28.154.131@KRBTEST.COM (short username kudu)
I20260811 09:10:08.508285 18700 webserver.cc:460] Webserver started at http://127.28.154.131:45035/ using document root <none> and password file <none>
I20260811 09:10:08.509240 18700 fs_manager.cc:359] Metadata directory not provided
I20260811 09:10:08.509450 18700 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:10:08.509927 18700 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 09:10:08 dist-test-slave-xt4k krb5kdc[17161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439408, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.131@KRBTEST.COM for HTTP/127.28.154.148@KRBTEST.COM
I20260811 09:10:08.529482 18700 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/minicluster-data/ts-2/data/instance:
uuid: "4302358d141d4eae8528512d0e44a19b"
format_stamp: "Formatted at 2026-08-11 09:10:08 on dist-test-slave-xt4k"
server_key: "0761cbdbf2cf2db8f0612ef660157f97"
server_key_iv: "73daf04195b479068af45acc42b7c682"
server_key_version: "kuduclusterkey@0"
I20260811 09:10:08.530611 18700 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/minicluster-data/ts-2/wal/instance:
uuid: "4302358d141d4eae8528512d0e44a19b"
format_stamp: "Formatted at 2026-08-11 09:10:08 on dist-test-slave-xt4k"
server_key: "0761cbdbf2cf2db8f0612ef660157f97"
server_key_iv: "73daf04195b479068af45acc42b7c682"
server_key_version: "kuduclusterkey@0"
I20260811 09:10:08.538317 18700 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.004s sys 0.004s
I20260811 09:10:08.558859 18724 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:10:08.560792 18700 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260811 09:10:08.561038 18700 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/minicluster-data/ts-2/wal
uuid: "4302358d141d4eae8528512d0e44a19b"
format_stamp: "Formatted at 2026-08-11 09:10:08 on dist-test-slave-xt4k"
server_key: "0761cbdbf2cf2db8f0612ef660157f97"
server_key_iv: "73daf04195b479068af45acc42b7c682"
server_key_version: "kuduclusterkey@0"
I20260811 09:10:08.561345 18700 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:10:08.584865 18700 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:10:08.590752 18700 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:10:08.591194 18700 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:10:08.593118 18700 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:10:08.595913 18700 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:10:08.596067 18700 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:10:08.596290 18700 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:10:08.596446 18700 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260811 09:10:08.682566 18700 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.131:43907
I20260811 09:10:08.682559 18920 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.131:43907 every 8 connection(s)
I20260811 09:10:08.684530 18700 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/minicluster-data/ts-2/data/info.pb
I20260811 09:10:08.690608 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 18700
I20260811 09:10:08.690883 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/minicluster-data/ts-2/wal/instance
I20260811 09:10:08.691429 29290 external_mini_cluster.cc:1428] Setting key 2d4be1f1d8e50792da4b04dc4a3f55bd
Aug 11 09:10:08 dist-test-slave-xt4k krb5kdc[17161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439408, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.131@KRBTEST.COM for kudu/127.28.154.190@KRBTEST.COM
I20260811 09:10:08.714412 18687 heartbeater.cc:503] Master 127.28.154.190:38573 was elected leader, sending a full tablet report...
I20260811 09:10:08.716073 18924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:10:08.688807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.131:53963 (local address 127.28.154.190:38573)
0811 09:10:08.689942 (+ 1135us) server_negotiation.cc:206] Beginning negotiation
0811 09:10:08.689946 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:10:08.691428 (+ 1482us) server_negotiation.cc:407] Connection header received
0811 09:10:08.695082 (+ 3654us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:10:08.695088 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:10:08.695178 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:10:08.695272 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:10:08.697430 (+ 2158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:10:08.698681 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:10:08.700970 (+ 2289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:10:08.701721 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:10:08.701832 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:10:08.707611 (+ 5779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:10:08.707622 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:10:08.707627 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:10:08.707724 (+ 97us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:10:08.711816 (+ 4092us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:10:08.712855 (+ 1039us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:10:08.712860 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:10:08.712862 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:10:08.713048 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:10:08.713630 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:10:08.713633 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:10:08.713636 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:10:08.714140 (+ 504us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:10:08.714296 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0811 09:10:08.715299 (+ 1003us) server_negotiation.cc:299] Negotiation successful
0811 09:10:08.715533 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1026,"thread_start_us":721,"threads_started":1}
I20260811 09:10:08.717358 18925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:10:08.689300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:38573 (local address 127.28.154.131:53963)
0811 09:10:08.691208 (+ 1908us) negotiation.cc:107] Waiting for socket to connect
0811 09:10:08.691275 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0811 09:10:08.694721 (+ 3446us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:10:08.695598 (+ 877us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:10:08.695608 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:10:08.696227 (+ 619us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:10:08.697240 (+ 1013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:10:08.697263 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:10:08.698838 (+ 1575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:10:08.698842 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:10:08.700723 (+ 1881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:10:08.700769 (+ 46us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:10:08.702064 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:10:08.702072 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:10:08.702401 (+ 329us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:10:08.703221 (+ 820us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:10:08.703249 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:10:08.707243 (+ 3994us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:10:08.711994 (+ 4751us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:10:08.712003 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:10:08.712021 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:10:08.712716 (+ 695us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:10:08.713186 (+ 470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:10:08.713190 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:10:08.713193 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:10:08.713496 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:10:08.714353 (+ 857us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:10:08.714360 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:10:08.714787 (+ 427us) client_negotiation.cc:769] Sending connection context
0811 09:10:08.715182 (+ 395us) client_negotiation.cc:240] Negotiation successful
0811 09:10:08.715906 (+ 724us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1489,"thread_start_us":1046,"threads_started":1}
I20260811 09:10:08.719089 18922 heartbeater.cc:347] Connected to a master server at 127.28.154.190:38573
I20260811 09:10:08.719496 18922 heartbeater.cc:464] Registering TS with master...
I20260811 09:10:08.720786 18922 heartbeater.cc:511] Master 127.28.154.190:38573 requested a full tablet report, sending...
I20260811 09:10:08.722824 18041 ts_manager.cc:194] Registered new tserver with Master: 4302358d141d4eae8528512d0e44a19b (127.28.154.131:43907)
I20260811 09:10:08.723934 18041 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.154.131@KRBTEST.COM'} at 127.28.154.131:53963
I20260811 09:10:08.734974 29290 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 09:10:08.761775 18924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:10:08.742763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57830 (local address 127.28.154.190:38573)
0811 09:10:08.743013 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0811 09:10:08.743020 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:10:08.743844 (+ 824us) server_negotiation.cc:407] Connection header received
0811 09:10:08.744047 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:10:08.744052 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:10:08.744180 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:10:08.744275 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:10:08.746633 (+ 2358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:10:08.747586 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:10:08.749622 (+ 2036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:10:08.750313 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:10:08.750438 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:10:08.754069 (+ 3631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:10:08.754075 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:10:08.754078 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:10:08.754117 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:10:08.757951 (+ 3834us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:10:08.759032 (+ 1081us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:10:08.759038 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:10:08.759040 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:10:08.759297 (+ 257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:10:08.759883 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:10:08.759887 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:10:08.759889 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:10:08.760323 (+ 434us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:10:08.760695 (+ 372us) server_negotiation.cc:1035] Waiting for connection context
0811 09:10:08.761139 (+ 444us) server_negotiation.cc:299] Negotiation successful
0811 09:10:08.761388 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260811 09:10:08.767583 18041 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57830:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 09:10:08.779467 18041 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 09:10:08.822643 18954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:10:08.797908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.131:43907 (local address 127.0.0.1:46648)
0811 09:10:08.801802 (+ 3894us) negotiation.cc:107] Waiting for socket to connect
0811 09:10:08.801847 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 09:10:08.802128 (+ 281us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:10:08.811051 (+ 8923us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:10:08.811058 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:10:08.811090 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:10:08.812399 (+ 1309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:10:08.812417 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:10:08.814979 (+ 2562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:10:08.814985 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:10:08.818706 (+ 3721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:10:08.818720 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:10:08.821365 (+ 2645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:10:08.821373 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:10:08.821662 (+ 289us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:10:08.821726 (+ 64us) client_negotiation.cc:769] Sending connection context
0811 09:10:08.821920 (+ 194us) client_negotiation.cc:240] Negotiation successful
0811 09:10:08.822139 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3491,"thread_start_us":3195,"threads_started":1}
I20260811 09:10:08.823366 18957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:10:08.798809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46648 (local address 127.28.154.131:43907)
0811 09:10:08.810117 (+ 11308us) server_negotiation.cc:206] Beginning negotiation
0811 09:10:08.810127 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 09:10:08.810201 (+ 74us) server_negotiation.cc:407] Connection header received
0811 09:10:08.810333 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:10:08.810344 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:10:08.810736 (+ 392us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:10:08.811024 (+ 288us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:10:08.812761 (+ 1737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:10:08.814805 (+ 2044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:10:08.818868 (+ 4063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:10:08.820601 (+ 1733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:10:08.820912 (+ 311us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:10:08.821117 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0811 09:10:08.822110 (+ 993us) server_negotiation.cc:299] Negotiation successful
0811 09:10:08.822517 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11105,"thread_start_us":10837,"threads_started":1}
I20260811 09:10:08.824064 18955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:10:08.802267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.130:35151 (local address 127.0.0.1:37144)
0811 09:10:08.810805 (+ 8538us) negotiation.cc:107] Waiting for socket to connect
0811 09:10:08.810829 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 09:10:08.811026 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:10:08.811804 (+ 778us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:10:08.811810 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:10:08.811837 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:10:08.813969 (+ 2132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:10:08.814000 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:10:08.816442 (+ 2442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:10:08.816447 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:10:08.819924 (+ 3477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:10:08.819940 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:10:08.822898 (+ 2958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:10:08.822905 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:10:08.823112 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:10:08.823143 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 09:10:08.823222 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 09:10:08.823451 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8318,"mutex_wait_us":2,"thread_start_us":5545,"threads_started":1}
I20260811 09:10:08.825177 18961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:10:08.803417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37144 (local address 127.28.154.130:35151)
0811 09:10:08.808032 (+ 4615us) server_negotiation.cc:206] Beginning negotiation
0811 09:10:08.808045 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 09:10:08.810956 (+ 2911us) server_negotiation.cc:407] Connection header received
0811 09:10:08.811239 (+ 283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:10:08.811247 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:10:08.811540 (+ 293us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:10:08.811788 (+ 248us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:10:08.814182 (+ 2394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:10:08.816181 (+ 1999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:10:08.820107 (+ 3926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:10:08.822243 (+ 2136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:10:08.822528 (+ 285us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:10:08.822700 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0811 09:10:08.824313 (+ 1613us) server_negotiation.cc:299] Negotiation successful
0811 09:10:08.824589 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4376,"thread_start_us":4047,"threads_started":1}
I20260811 09:10:08.825714 18962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:10:08.798805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.129:39433 (local address 127.0.0.1:33252)
0811 09:10:08.808155 (+ 9350us) negotiation.cc:107] Waiting for socket to connect
0811 09:10:08.808182 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 09:10:08.808409 (+ 227us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:10:08.813148 (+ 4739us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:10:08.813152 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:10:08.813180 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:10:08.814798 (+ 1618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:10:08.814815 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:10:08.817889 (+ 3074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:10:08.817895 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:10:08.821175 (+ 3280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:10:08.821189 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:10:08.825007 (+ 3818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:10:08.825013 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:10:08.825199 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:10:08.825234 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 09:10:08.825337 (+ 103us) client_negotiation.cc:240] Negotiation successful
0811 09:10:08.825471 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9145,"thread_start_us":13147,"threads_started":1}
I20260811 09:10:08.826375 18959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:10:08.799492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33252 (local address 127.28.154.129:39433)
0811 09:10:08.812380 (+ 12888us) server_negotiation.cc:206] Beginning negotiation
0811 09:10:08.812391 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 09:10:08.812420 (+ 29us) server_negotiation.cc:407] Connection header received
0811 09:10:08.812517 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:10:08.812525 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:10:08.812820 (+ 295us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:10:08.813495 (+ 675us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:10:08.815139 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:10:08.817664 (+ 2525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:10:08.821370 (+ 3706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:10:08.824712 (+ 3342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:10:08.825060 (+ 348us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:10:08.825276 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0811 09:10:08.825377 (+ 101us) server_negotiation.cc:299] Negotiation successful
0811 09:10:08.825591 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12625,"thread_start_us":12438,"threads_started":1}
I20260811 09:10:08.829635 18808 tablet_service.cc:1467] Processing CreateTablet for tablet 9b4a5e5024c445d69602a76df25aacb0 (DEFAULT_TABLE table=test-table [id=4cd1a7825c57427f9f934c9293d78477]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:10:08.830897 18575 tablet_service.cc:1467] Processing CreateTablet for tablet 9b4a5e5024c445d69602a76df25aacb0 (DEFAULT_TABLE table=test-table [id=4cd1a7825c57427f9f934c9293d78477]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:10:08.831275 18316 tablet_service.cc:1467] Processing CreateTablet for tablet 9b4a5e5024c445d69602a76df25aacb0 (DEFAULT_TABLE table=test-table [id=4cd1a7825c57427f9f934c9293d78477]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:10:08.831966 18808 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b4a5e5024c445d69602a76df25aacb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:10:08.832648 18575 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b4a5e5024c445d69602a76df25aacb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:10:08.833346 18316 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9b4a5e5024c445d69602a76df25aacb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:10:08.849502 18964 tablet_bootstrap.cc:492] T 9b4a5e5024c445d69602a76df25aacb0 P f3c7f757dc6448f98f5106989efea337: Bootstrap starting.
I20260811 09:10:08.855126 18964 tablet_bootstrap.cc:654] T 9b4a5e5024c445d69602a76df25aacb0 P f3c7f757dc6448f98f5106989efea337: Neither blocks nor log segments found. Creating new log.
I20260811 09:10:08.855491 18967 tablet_bootstrap.cc:492] T 9b4a5e5024c445d69602a76df25aacb0 P 4302358d141d4eae8528512d0e44a19b: Bootstrap starting.
I20260811 09:10:08.858028 18964 log.cc:824] T 9b4a5e5024c445d69602a76df25aacb0 P f3c7f757dc6448f98f5106989efea337: Log is configured to *not* fsync() on all Append() calls
I20260811 09:10:08.858111 18968 tablet_bootstrap.cc:492] T 9b4a5e5024c445d69602a76df25aacb0 P 55110c3cf110404a8a7fa2591be4411f: Bootstrap starting.
I20260811 09:10:08.862155 18967 tablet_bootstrap.cc:654] T 9b4a5e5024c445d69602a76df25aacb0 P 4302358d141d4eae8528512d0e44a19b: Neither blocks nor log segments found. Creating new log.
I20260811 09:10:08.863903 18968 tablet_bootstrap.cc:654] T 9b4a5e5024c445d69602a76df25aacb0 P 55110c3cf110404a8a7fa2591be4411f: Neither blocks nor log segments found. Creating new log.
I20260811 09:10:08.864898 18967 log.cc:824] T 9b4a5e5024c445d69602a76df25aacb0 P 4302358d141d4eae8528512d0e44a19b: Log is configured to *not* fsync() on all Append() calls
I20260811 09:10:08.865429 18964 tablet_bootstrap.cc:492] T 9b4a5e5024c445d69602a76df25aacb0 P f3c7f757dc6448f98f5106989efea337: No bootstrap required, opened a new log
I20260811 09:10:08.865890 18964 ts_tablet_manager.cc:1397] T 9b4a5e5024c445d69602a76df25aacb0 P f3c7f757dc6448f98f5106989efea337: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.012s
I20260811 09:10:08.866243 18968 log.cc:824] T 9b4a5e5024c445d69602a76df25aacb0 P 55110c3cf110404a8a7fa2591be4411f: Log is configured to *not* fsync() on all Append() calls
I20260811 09:10:08.870236 18967 tablet_bootstrap.cc:492] T 9b4a5e5024c445d69602a76df25aacb0 P 4302358d141d4eae8528512d0e44a19b: No bootstrap required, opened a new log
I20260811 09:10:08.870628 18967 ts_tablet_manager.cc:1397] T 9b4a5e5024c445d69602a76df25aacb0 P 4302358d141d4eae8528512d0e44a19b: Time spent bootstrapping tablet: real 0.015s user 0.006s sys 0.008s
I20260811 09:10:08.871625 18968 tablet_bootstrap.cc:492] T 9b4a5e5024c445d69602a76df25aacb0 P 55110c3cf110404a8a7fa2591be4411f: No bootstrap required, opened a new log
I20260811 09:10:08.871996 18968 ts_tablet_manager.cc:1397] T 9b4a5e5024c445d69602a76df25aacb0 P 55110c3cf110404a8a7fa2591be4411f: Time spent bootstrapping tablet: real 0.014s user 0.003s sys 0.008s
I20260811 09:10:08.882103 18964 raft_consensus.cc:348] T 9b4a5e5024c445d69602a76df25aacb0 P f3c7f757dc6448f98f5106989efea337 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4302358d141d4eae8528512d0e44a19b" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 43907 } } peers { permanent_uuid: "f3c7f757dc6448f98f5106989efea337" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 35151 } } peers { permanent_uuid: "55110c3cf110404a8a7fa2591be4411f" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 39433 } }
I20260811 09:10:08.882972 18964 raft_consensus.cc:374] T 9b4a5e5024c445d69602a76df25aacb0 P f3c7f757dc6448f98f5106989efea337 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:10:08.883186 18964 raft_consensus.cc:729] T 9b4a5e5024c445d69602a76df25aacb0 P f3c7f757dc6448f98f5106989efea337 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3c7f757dc6448f98f5106989efea337, State: Initialized, Role: FOLLOWER
I20260811 09:10:08.884166 18968 raft_consensus.cc:348] T 9b4a5e5024c445d69602a76df25aacb0 P 55110c3cf110404a8a7fa2591be4411f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4302358d141d4eae8528512d0e44a19b" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 43907 } } peers { permanent_uuid: "f3c7f757dc6448f98f5106989efea337" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 35151 } } peers { permanent_uuid: "55110c3cf110404a8a7fa2591be4411f" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 39433 } }
I20260811 09:10:08.884177 18964 consensus_queue.cc:260] T 9b4a5e5024c445d69602a76df25aacb0 P f3c7f757dc6448f98f5106989efea337 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4302358d141d4eae8528512d0e44a19b" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 43907 } } peers { permanent_uuid: "f3c7f757dc6448f98f5106989efea337" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 35151 } } peers { permanent_uuid: "55110c3cf110404a8a7fa2591be4411f" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 39433 } }
I20260811 09:10:08.884728 18968 raft_consensus.cc:374] T 9b4a5e5024c445d69602a76df25aacb0 P 55110c3cf110404a8a7fa2591be4411f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:10:08.884886 18968 raft_consensus.cc:729] T 9b4a5e5024c445d69602a76df25aacb0 P 55110c3cf110404a8a7fa2591be4411f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55110c3cf110404a8a7fa2591be4411f, State: Initialized, Role: FOLLOWER
I20260811 09:10:08.885756 18968 consensus_queue.cc:260] T 9b4a5e5024c445d69602a76df25aacb0 P 55110c3cf110404a8a7fa2591be4411f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4302358d141d4eae8528512d0e44a19b" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 43907 } } peers { permanent_uuid: "f3c7f757dc6448f98f5106989efea337" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 35151 } } peers { permanent_uuid: "55110c3cf110404a8a7fa2591be4411f" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 39433 } }
I20260811 09:10:08.887126 18967 raft_consensus.cc:348] T 9b4a5e5024c445d69602a76df25aacb0 P 4302358d141d4eae8528512d0e44a19b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4302358d141d4eae8528512d0e44a19b" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 43907 } } peers { permanent_uuid: "f3c7f757dc6448f98f5106989efea337" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 35151 } } peers { permanent_uuid: "55110c3cf110404a8a7fa2591be4411f" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 39433 } }
I20260811 09:10:08.887750 18967 raft_consensus.cc:374] T 9b4a5e5024c445d69602a76df25aacb0 P 4302358d141d4eae8528512d0e44a19b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:10:08.887934 18967 raft_consensus.cc:729] T 9b4a5e5024c445d69602a76df25aacb0 P 4302358d141d4eae8528512d0e44a19b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4302358d141d4eae8528512d0e44a19b, State: Initialized, Role: FOLLOWER
I20260811 09:10:08.888893 18968 ts_tablet_manager.cc:1428] T 9b4a5e5024c445d69602a76df25aacb0 P 55110c3cf110404a8a7fa2591be4411f: Time spent starting tablet: real 0.017s user 0.008s sys 0.009s
I20260811 09:10:08.889199 18967 consensus_queue.cc:260] T 9b4a5e5024c445d69602a76df25aacb0 P 4302358d141d4eae8528512d0e44a19b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4302358d141d4eae8528512d0e44a19b" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 43907 } } peers { permanent_uuid: "f3c7f757dc6448f98f5106989efea337" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 35151 } } peers { permanent_uuid: "55110c3cf110404a8a7fa2591be4411f" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 39433 } }
I20260811 09:10:08.890070 18964 ts_tablet_manager.cc:1428] T 9b4a5e5024c445d69602a76df25aacb0 P f3c7f757dc6448f98f5106989efea337: Time spent starting tablet: real 0.024s user 0.020s sys 0.000s
I20260811 09:10:08.891839 18922 heartbeater.cc:503] Master 127.28.154.190:38573 was elected leader, sending a full tablet report...
I20260811 09:10:08.894049 18967 ts_tablet_manager.cc:1428] T 9b4a5e5024c445d69602a76df25aacb0 P 4302358d141d4eae8528512d0e44a19b: Time spent starting tablet: real 0.023s user 0.016s sys 0.006s
W20260811 09:10:08.923549 18688 tablet.cc:2369] T 9b4a5e5024c445d69602a76df25aacb0 P f3c7f757dc6448f98f5106989efea337: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 09:10:08.937740 18923 tablet.cc:2369] T 9b4a5e5024c445d69602a76df25aacb0 P 4302358d141d4eae8528512d0e44a19b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 09:10:08.987928 18973 raft_consensus.cc:482] T 9b4a5e5024c445d69602a76df25aacb0 P f3c7f757dc6448f98f5106989efea337 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:10:08.988528 18973 raft_consensus.cc:504] T 9b4a5e5024c445d69602a76df25aacb0 P f3c7f757dc6448f98f5106989efea337 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4302358d141d4eae8528512d0e44a19b" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 43907 } } peers { permanent_uuid: "f3c7f757dc6448f98f5106989efea337" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 35151 } } peers { permanent_uuid: "55110c3cf110404a8a7fa2591be4411f" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 39433 } }
I20260811 09:10:08.991290 18973 leader_election.cc:288] T 9b4a5e5024c445d69602a76df25aacb0 P f3c7f757dc6448f98f5106989efea337 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4302358d141d4eae8528512d0e44a19b (127.28.154.131:43907), 55110c3cf110404a8a7fa2591be4411f (127.28.154.129:39433)
I20260811 09:10:09.003059 18959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:10:08.992028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.130:53029 (local address 127.28.154.129:39433)
0811 09:10:08.992295 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0811 09:10:08.992302 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:10:08.994804 (+ 2502us) server_negotiation.cc:407] Connection header received
0811 09:10:08.995003 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:10:08.995008 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:10:08.995072 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:10:08.995151 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:10:08.996399 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:10:08.997490 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:10:09.000772 (+ 3282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:10:09.001845 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:10:09.002011 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:10:09.002086 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 09:10:09.002605 (+ 519us) server_negotiation.cc:299] Negotiation successful
0811 09:10:09.002738 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 09:10:09.003140 18957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:10:08.992130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.130:53329 (local address 127.28.154.131:43907)
0811 09:10:08.992435 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0811 09:10:08.992441 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:10:08.994536 (+ 2095us) server_negotiation.cc:407] Connection header received
0811 09:10:08.994868 (+ 332us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:10:08.994873 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:10:08.994941 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:10:08.995023 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:10:08.996246 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:10:08.997349 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:10:09.000446 (+ 3097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:10:09.001422 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:10:09.001604 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:10:09.001843 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0811 09:10:09.002498 (+ 655us) server_negotiation.cc:299] Negotiation successful
0811 09:10:09.002773 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260811 09:10:09.003465 18977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:10:08.992237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.129:39433 (local address 127.28.154.130:53029)
0811 09:10:08.994672 (+ 2435us) negotiation.cc:107] Waiting for socket to connect
0811 09:10:08.994688 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 09:10:08.994861 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:10:08.995182 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:10:08.995187 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:10:08.995211 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:10:08.996178 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:10:08.996191 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:10:08.997670 (+ 1479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:10:08.997674 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:10:09.000573 (+ 2899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:10:09.000586 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:10:09.002220 (+ 1634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:10:09.002227 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:10:09.002424 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:10:09.002460 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 09:10:09.002554 (+ 94us) client_negotiation.cc:240] Negotiation successful
0811 09:10:09.002721 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2262,"mutex_wait_us":1,"thread_start_us":1408,"threads_started":1}
I20260811 09:10:09.003547 18979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:10:08.991645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.131:43907 (local address 127.28.154.130:53329)
0811 09:10:08.994377 (+ 2732us) negotiation.cc:107] Waiting for socket to connect
0811 09:10:08.994421 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 09:10:08.994671 (+ 250us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:10:08.995058 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:10:08.995063 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:10:08.995090 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:10:08.996088 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:10:08.996104 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:10:08.997504 (+ 1400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:10:08.997509 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:10:09.000253 (+ 2744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:10:09.000268 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:10:09.001841 (+ 1573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:10:09.001863 (+ 22us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:10:09.002218 (+ 355us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:10:09.002259 (+ 41us) client_negotiation.cc:769] Sending connection context
0811 09:10:09.002441 (+ 182us) client_negotiation.cc:240] Negotiation successful
0811 09:10:09.002637 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2550,"mutex_wait_us":132,"thread_start_us":2544,"threads_started":1}
I20260811 09:10:09.005465 18360 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b4a5e5024c445d69602a76df25aacb0" candidate_uuid: "f3c7f757dc6448f98f5106989efea337" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55110c3cf110404a8a7fa2591be4411f" is_pre_election: true
I20260811 09:10:09.005887 18836 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b4a5e5024c445d69602a76df25aacb0" candidate_uuid: "f3c7f757dc6448f98f5106989efea337" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4302358d141d4eae8528512d0e44a19b" is_pre_election: true
I20260811 09:10:09.006702 18360 raft_consensus.cc:2454] T 9b4a5e5024c445d69602a76df25aacb0 P 55110c3cf110404a8a7fa2591be4411f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c7f757dc6448f98f5106989efea337 in term 0.
I20260811 09:10:09.006680 18836 raft_consensus.cc:2454] T 9b4a5e5024c445d69602a76df25aacb0 P 4302358d141d4eae8528512d0e44a19b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c7f757dc6448f98f5106989efea337 in term 0.
I20260811 09:10:09.007737 18492 leader_election.cc:302] T 9b4a5e5024c445d69602a76df25aacb0 P f3c7f757dc6448f98f5106989efea337 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4302358d141d4eae8528512d0e44a19b, f3c7f757dc6448f98f5106989efea337; no voters:
I20260811 09:10:09.008477 18973 raft_consensus.cc:2781] T 9b4a5e5024c445d69602a76df25aacb0 P f3c7f757dc6448f98f5106989efea337 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 09:10:09.008796 18973 raft_consensus.cc:482] T 9b4a5e5024c445d69602a76df25aacb0 P f3c7f757dc6448f98f5106989efea337 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 09:10:09.009020 18973 raft_consensus.cc:3037] T 9b4a5e5024c445d69602a76df25aacb0 P f3c7f757dc6448f98f5106989efea337 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:10:09.012516 18973 raft_consensus.cc:504] T 9b4a5e5024c445d69602a76df25aacb0 P f3c7f757dc6448f98f5106989efea337 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4302358d141d4eae8528512d0e44a19b" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 43907 } } peers { permanent_uuid: "f3c7f757dc6448f98f5106989efea337" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 35151 } } peers { permanent_uuid: "55110c3cf110404a8a7fa2591be4411f" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 39433 } }
I20260811 09:10:09.013409 18973 leader_election.cc:288] T 9b4a5e5024c445d69602a76df25aacb0 P f3c7f757dc6448f98f5106989efea337 [CANDIDATE]: Term 1 election: Requested vote from peers 4302358d141d4eae8528512d0e44a19b (127.28.154.131:43907), 55110c3cf110404a8a7fa2591be4411f (127.28.154.129:39433)
I20260811 09:10:09.014083 18360 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b4a5e5024c445d69602a76df25aacb0" candidate_uuid: "f3c7f757dc6448f98f5106989efea337" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55110c3cf110404a8a7fa2591be4411f"
I20260811 09:10:09.014211 18836 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9b4a5e5024c445d69602a76df25aacb0" candidate_uuid: "f3c7f757dc6448f98f5106989efea337" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4302358d141d4eae8528512d0e44a19b"
I20260811 09:10:09.014499 18836 raft_consensus.cc:3037] T 9b4a5e5024c445d69602a76df25aacb0 P 4302358d141d4eae8528512d0e44a19b [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:10:09.014482 18360 raft_consensus.cc:3037] T 9b4a5e5024c445d69602a76df25aacb0 P 55110c3cf110404a8a7fa2591be4411f [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:10:09.018846 18836 raft_consensus.cc:2454] T 9b4a5e5024c445d69602a76df25aacb0 P 4302358d141d4eae8528512d0e44a19b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c7f757dc6448f98f5106989efea337 in term 1.
I20260811 09:10:09.019290 18360 raft_consensus.cc:2454] T 9b4a5e5024c445d69602a76df25aacb0 P 55110c3cf110404a8a7fa2591be4411f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c7f757dc6448f98f5106989efea337 in term 1.
I20260811 09:10:09.019904 18492 leader_election.cc:302] T 9b4a5e5024c445d69602a76df25aacb0 P f3c7f757dc6448f98f5106989efea337 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4302358d141d4eae8528512d0e44a19b, f3c7f757dc6448f98f5106989efea337; no voters:
I20260811 09:10:09.020457 18973 raft_consensus.cc:2781] T 9b4a5e5024c445d69602a76df25aacb0 P f3c7f757dc6448f98f5106989efea337 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:10:09.021575 18973 raft_consensus.cc:686] T 9b4a5e5024c445d69602a76df25aacb0 P f3c7f757dc6448f98f5106989efea337 [term 1 LEADER]: Becoming Leader. State: Replica: f3c7f757dc6448f98f5106989efea337, State: Running, Role: LEADER
I20260811 09:10:09.022241 18973 consensus_queue.cc:237] T 9b4a5e5024c445d69602a76df25aacb0 P f3c7f757dc6448f98f5106989efea337 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4302358d141d4eae8528512d0e44a19b" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 43907 } } peers { permanent_uuid: "f3c7f757dc6448f98f5106989efea337" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 35151 } } peers { permanent_uuid: "55110c3cf110404a8a7fa2591be4411f" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 39433 } }
I20260811 09:10:09.029397 18040 catalog_manager.cc:5495] T 9b4a5e5024c445d69602a76df25aacb0 P f3c7f757dc6448f98f5106989efea337 reported cstate change: term changed from 0 to 1, leader changed from <none> to f3c7f757dc6448f98f5106989efea337 (127.28.154.130). New cstate: current_term: 1 leader_uuid: "f3c7f757dc6448f98f5106989efea337" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4302358d141d4eae8528512d0e44a19b" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 43907 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c7f757dc6448f98f5106989efea337" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 35151 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "55110c3cf110404a8a7fa2591be4411f" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 39433 } health_report { overall_health: UNKNOWN } } }
W20260811 09:10:09.062449 18434 tablet.cc:2369] T 9b4a5e5024c445d69602a76df25aacb0 P 55110c3cf110404a8a7fa2591be4411f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 09:10:09.067587 18961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:10:09.060236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37158 (local address 127.28.154.130:35151)
0811 09:10:09.060498 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0811 09:10:09.060504 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:10:09.060590 (+ 86us) server_negotiation.cc:407] Connection header received
0811 09:10:09.060815 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:10:09.060818 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:10:09.060899 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:10:09.061036 (+ 137us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:10:09.061619 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:10:09.062685 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:10:09.064624 (+ 1939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:10:09.065337 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:10:09.065455 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:10:09.066251 (+ 796us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:10:09.066867 (+ 616us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:10:09.067022 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0811 09:10:09.067128 (+ 106us) server_negotiation.cc:299] Negotiation successful
0811 09:10:09.067214 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119}
I20260811 09:10:09.078734 18836 raft_consensus.cc:1261] T 9b4a5e5024c445d69602a76df25aacb0 P 4302358d141d4eae8528512d0e44a19b [term 1 FOLLOWER]: Refusing update from remote peer f3c7f757dc6448f98f5106989efea337: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:10:09.078756 18360 raft_consensus.cc:1261] T 9b4a5e5024c445d69602a76df25aacb0 P 55110c3cf110404a8a7fa2591be4411f [term 1 FOLLOWER]: Refusing update from remote peer f3c7f757dc6448f98f5106989efea337: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:10:09.079896 18980 consensus_queue.cc:1035] T 9b4a5e5024c445d69602a76df25aacb0 P f3c7f757dc6448f98f5106989efea337 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4302358d141d4eae8528512d0e44a19b" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 43907 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:10:09.080610 18973 consensus_queue.cc:1035] T 9b4a5e5024c445d69602a76df25aacb0 P f3c7f757dc6448f98f5106989efea337 [LEADER]: Connected to new peer: Peer: permanent_uuid: "55110c3cf110404a8a7fa2591be4411f" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 39433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:10:09.096683 18985 mvcc.cc:205] Tried to move back new op lower bound from 7317255819574210560 to 7317255819360268288. Current Snapshot: MvccSnapshot[applied={T|T < 7317255819574210560}]
I20260811 09:10:09.097028 18987 mvcc.cc:205] Tried to move back new op lower bound from 7317255819574210560 to 7317255819360268288. Current Snapshot: MvccSnapshot[applied={T|T < 7317255819574210560}]
I20260811 09:10:09.117291 18039 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57830:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:10:09.117920 18039 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57830:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:10:09.123718 18039 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6da0e5689c6d485387173849aa7f4ff4: Sending DeleteTablet for 3 replicas of tablet 9b4a5e5024c445d69602a76df25aacb0
I20260811 09:10:09.125275 18808 tablet_service.cc:1514] Processing DeleteTablet for tablet 9b4a5e5024c445d69602a76df25aacb0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:10:09 UTC) from {username='kudu', principal='kudu/127.28.154.190@KRBTEST.COM'} at 127.0.0.1:46648
I20260811 09:10:09.125308 18575 tablet_service.cc:1514] Processing DeleteTablet for tablet 9b4a5e5024c445d69602a76df25aacb0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:10:09 UTC) from {username='kudu', principal='kudu/127.28.154.190@KRBTEST.COM'} at 127.0.0.1:37144
I20260811 09:10:09.125699 18316 tablet_service.cc:1514] Processing DeleteTablet for tablet 9b4a5e5024c445d69602a76df25aacb0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:10:09 UTC) from {username='kudu', principal='kudu/127.28.154.190@KRBTEST.COM'} at 127.0.0.1:33252
I20260811 09:10:09.128496 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 18190
I20260811 09:10:09.138005 19004 tablet_replica.cc:330] T 9b4a5e5024c445d69602a76df25aacb0 P 55110c3cf110404a8a7fa2591be4411f: stopping tablet replica
I20260811 09:10:09.138017 19002 tablet_replica.cc:330] T 9b4a5e5024c445d69602a76df25aacb0 P 4302358d141d4eae8528512d0e44a19b: stopping tablet replica
I20260811 09:10:09.138584 19002 raft_consensus.cc:2229] T 9b4a5e5024c445d69602a76df25aacb0 P 4302358d141d4eae8528512d0e44a19b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:10:09.139348 19002 raft_consensus.cc:2258] T 9b4a5e5024c445d69602a76df25aacb0 P 4302358d141d4eae8528512d0e44a19b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:10:09.139613 19004 raft_consensus.cc:2229] T 9b4a5e5024c445d69602a76df25aacb0 P 55110c3cf110404a8a7fa2591be4411f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:10:09.141067 19004 raft_consensus.cc:2258] T 9b4a5e5024c445d69602a76df25aacb0 P 55110c3cf110404a8a7fa2591be4411f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:10:09.141688 19002 ts_tablet_manager.cc:1905] T 9b4a5e5024c445d69602a76df25aacb0 P 4302358d141d4eae8528512d0e44a19b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:10:09.143198 19004 ts_tablet_manager.cc:1905] T 9b4a5e5024c445d69602a76df25aacb0 P 55110c3cf110404a8a7fa2591be4411f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:10:09.152052 19004 ts_tablet_manager.cc:1918] T 9b4a5e5024c445d69602a76df25aacb0 P 55110c3cf110404a8a7fa2591be4411f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:10:09.152472 19004 log.cc:1196] T 9b4a5e5024c445d69602a76df25aacb0 P 55110c3cf110404a8a7fa2591be4411f: Deleting WAL directory at /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/minicluster-data/ts-0/wal/wals/9b4a5e5024c445d69602a76df25aacb0
I20260811 09:10:09.152953 19002 ts_tablet_manager.cc:1918] T 9b4a5e5024c445d69602a76df25aacb0 P 4302358d141d4eae8528512d0e44a19b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:10:09.153416 19002 log.cc:1196] T 9b4a5e5024c445d69602a76df25aacb0 P 4302358d141d4eae8528512d0e44a19b: Deleting WAL directory at /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/minicluster-data/ts-2/wal/wals/9b4a5e5024c445d69602a76df25aacb0
I20260811 09:10:09.153482 19004 ts_tablet_manager.cc:1939] T 9b4a5e5024c445d69602a76df25aacb0 P 55110c3cf110404a8a7fa2591be4411f: Deleting consensus metadata
I20260811 09:10:09.154310 19002 ts_tablet_manager.cc:1939] T 9b4a5e5024c445d69602a76df25aacb0 P 4302358d141d4eae8528512d0e44a19b: Deleting consensus metadata
I20260811 09:10:09.156098 18025 catalog_manager.cc:4841] TS 55110c3cf110404a8a7fa2591be4411f (127.28.154.129:39433): tablet 9b4a5e5024c445d69602a76df25aacb0 (table test-table [id=4cd1a7825c57427f9f934c9293d78477]) successfully deleted
I20260811 09:10:09.156847 18023 catalog_manager.cc:4841] TS 4302358d141d4eae8528512d0e44a19b (127.28.154.131:43907): tablet 9b4a5e5024c445d69602a76df25aacb0 (table test-table [id=4cd1a7825c57427f9f934c9293d78477]) successfully deleted
I20260811 09:10:09.161075 19003 tablet_replica.cc:330] T 9b4a5e5024c445d69602a76df25aacb0 P f3c7f757dc6448f98f5106989efea337: stopping tablet replica
Aug 11 09:10:09 dist-test-slave-xt4k krb5kdc[17161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439403, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.154.129@KRBTEST.COM
I20260811 09:10:09.161763 19003 raft_consensus.cc:2229] T 9b4a5e5024c445d69602a76df25aacb0 P f3c7f757dc6448f98f5106989efea337 [term 1 LEADER]: Raft consensus shutting down.
I20260811 09:10:09.162946 19003 raft_consensus.cc:2258] T 9b4a5e5024c445d69602a76df25aacb0 P f3c7f757dc6448f98f5106989efea337 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:10:09.165699 19003 ts_tablet_manager.cc:1905] T 9b4a5e5024c445d69602a76df25aacb0 P f3c7f757dc6448f98f5106989efea337: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:10:09.170660 18959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:10:09.130274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33264 (local address 127.28.154.129:39433)
0811 09:10:09.130476 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0811 09:10:09.130482 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:10:09.143869 (+ 13387us) server_negotiation.cc:407] Connection header received
0811 09:10:09.144402 (+ 533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:10:09.144408 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:10:09.144504 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:10:09.144580 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:10:09.146401 (+ 1821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:10:09.147561 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:10:09.150054 (+ 2493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:10:09.150808 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:10:09.150915 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:10:09.162863 (+ 11948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:10:09.162880 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:10:09.162895 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:10:09.162935 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:10:09.167099 (+ 4164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:10:09.167883 (+ 784us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:10:09.167891 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:10:09.167894 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:10:09.168100 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:10:09.168579 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:10:09.168584 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:10:09.168586 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:10:09.169153 (+ 567us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:10:09.169618 (+ 465us) server_negotiation.cc:1035] Waiting for connection context
0811 09:10:09.169823 (+ 205us) server_negotiation.cc:299] Negotiation successful
0811 09:10:09.170117 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 09:10:09.171782 18427 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:10:09.178894 19003 ts_tablet_manager.cc:1918] T 9b4a5e5024c445d69602a76df25aacb0 P f3c7f757dc6448f98f5106989efea337: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:10:09.179217 19003 log.cc:1196] T 9b4a5e5024c445d69602a76df25aacb0 P f3c7f757dc6448f98f5106989efea337: Deleting WAL directory at /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786439250192320-29290-0/minicluster-data/ts-1/wal/wals/9b4a5e5024c445d69602a76df25aacb0
I20260811 09:10:09.179992 19003 ts_tablet_manager.cc:1939] T 9b4a5e5024c445d69602a76df25aacb0 P f3c7f757dc6448f98f5106989efea337: Deleting consensus metadata
I20260811 09:10:09.182406 18023 catalog_manager.cc:4841] TS f3c7f757dc6448f98f5106989efea337 (127.28.154.130:35151): tablet 9b4a5e5024c445d69602a76df25aacb0 (table test-table [id=4cd1a7825c57427f9f934c9293d78477]) successfully deleted
I20260811 09:10:09.268676 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 18190
I20260811 09:10:09.304055 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 18446
Aug 11 09:10:09 dist-test-slave-xt4k krb5kdc[17161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439403, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.154.130@KRBTEST.COM
I20260811 09:10:09.325809 18961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:10:09.305070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37174 (local address 127.28.154.130:35151)
0811 09:10:09.305250 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0811 09:10:09.305255 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:10:09.305311 (+ 56us) server_negotiation.cc:407] Connection header received
0811 09:10:09.305503 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:10:09.305507 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:10:09.305581 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:10:09.305661 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:10:09.308248 (+ 2587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:10:09.309565 (+ 1317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:10:09.311717 (+ 2152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:10:09.312413 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:10:09.312589 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:10:09.318389 (+ 5800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:10:09.318410 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:10:09.318429 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:10:09.318494 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:10:09.321967 (+ 3473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:10:09.322917 (+ 950us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:10:09.322925 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:10:09.322927 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:10:09.323240 (+ 313us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:10:09.323649 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:10:09.323653 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:10:09.323655 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:10:09.324417 (+ 762us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:10:09.324895 (+ 478us) server_negotiation.cc:1035] Waiting for connection context
0811 09:10:09.325187 (+ 292us) server_negotiation.cc:299] Negotiation successful
0811 09:10:09.325440 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 09:10:09.326609 18676 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:10:09.438243 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 18446
I20260811 09:10:09.465369 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 18700
Aug 11 09:10:09 dist-test-slave-xt4k krb5kdc[17161](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439403, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.154.131@KRBTEST.COM
I20260811 09:10:09.481304 18957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:10:09.466130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46664 (local address 127.28.154.131:43907)
0811 09:10:09.466306 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 09:10:09.466310 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:10:09.466330 (+ 20us) server_negotiation.cc:407] Connection header received
0811 09:10:09.466456 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:10:09.466461 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:10:09.466534 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:10:09.466628 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:10:09.467875 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:10:09.468745 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:10:09.470005 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:10:09.470383 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:10:09.470480 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:10:09.475248 (+ 4768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:10:09.475266 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:10:09.475286 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:10:09.475325 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:10:09.478349 (+ 3024us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:10:09.479065 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:10:09.479074 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:10:09.479077 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:10:09.479331 (+ 254us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:10:09.479752 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:10:09.479756 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:10:09.479759 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:10:09.480182 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:10:09.480475 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0811 09:10:09.480850 (+ 375us) server_negotiation.cc:299] Negotiation successful
0811 09:10:09.481037 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 09:10:09.481977 18914 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:10:09.573326 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 18700
I20260811 09:10:09.600845 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 17992
I20260811 09:10:09.602028 18089 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:10:09.716419 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 17992
I20260811 09:10:09.748119 29290 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260811 09:10:10.094228 29290 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260811 09:10:10.094533 29290 mini_ranger.cc:66] Stopping Ranger...
I20260811 09:10:10.440745 29290 mini_ranger.cc:68] Stopped Ranger
2026-08-11 09:10:10.441 UTC [17223] LOG: received smart shutdown request
2026-08-11 09:10:10.445 UTC [17223] LOG: background worker "logical replication launcher" (PID 17243) exited with exit code 1
2026-08-11 09:10:10.445 UTC [17233] LOG: shutting down
2026-08-11 09:10:10.623 UTC [17223] LOG: database system is shut down
2026-08-11T09:10:10Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (54556 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:10:10 dist-test-slave-xt4k krb5kdc[19035](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:10:10 dist-test-slave-xt4k krb5kdc[19035](info): set up 2 sockets
Aug 11 09:10:10 dist-test-slave-xt4k krb5kdc[19035](info): commencing operation
krb5kdc: starting...
W20260811 09:10:12.024358 29290 mini_kdc.cc:121] Time spent starting KDC: real 1.254s user 0.004s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/krb5kdc/test-user.keytab.
Aug 11 09:10:12 dist-test-slave-xt4k krb5kdc[19035](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439412, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:10:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:10:12Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.28.154.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.28.154.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.28.154.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/krb5kdc/rangeradmin_127.28.154.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.28.154.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/krb5kdc/rangeradmin_127.28.154.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.28.154.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.28.154.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.28.154.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/krb5kdc/rangerlookup_127.28.154.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.28.154.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/krb5kdc/rangerlookup_127.28.154.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.28.154.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.28.154.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.28.154.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/krb5kdc/HTTP_127.28.154.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.28.154.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/krb5kdc/HTTP_127.28.154.148@KRBTEST.COM.keytab.
I20260811 09:10:12.429239 29290 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-tasklh2cYx/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/postgres -l logfile start
2026-08-11 09:10:15.106 UTC [19089] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-11 09:10:15.106 UTC [19089] LOG: listening on IPv4 address "127.28.154.148", port 44403
2026-08-11 09:10:15.109 UTC [19089] LOG: listening on Unix socket "/tmp/.s.PGSQL.44403"
2026-08-11 09:10:15.132 UTC [19093] LOG: database system was shut down at 2026-08-11 09:10:13 UTC
2026-08-11 09:10:15.136 UTC [19089] LOG: database system is ready to accept connections
I20260811 09:10:16.287294 29290 mini_postgres.cc:96] Postgres bound to 44403
2026-08-11 09:10:16.310 UTC [19115] FATAL: database "slave" does not exist
127.28.154.148:44403 - accepting connections
I20260811 09:10:16.311625 29290 mini_ranger.cc:161] Starting Ranger...
I20260811 09:10:16.344172 29290 mini_ranger.cc:84] Created miniranger Postgres user
I20260811 09:10:16.887079 29290 mini_ranger.cc:87] Created ranger Postgres database
I20260811 09:10:16.887187 29290 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ranger-admin
2026-08-11 09:10:17,208 [I] DB FLAVOR :POSTGRES
2026-08-11 09:10:17,208 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 09:10:17,209 [I] Checking connection..
2026-08-11 09:10:17,209 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklh2cYx/build/asan/bin/postgresql.jar:/tmp/dist-test-tasklh2cYx/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.154.148:44403/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-11 09:10:17,475 [I] Checking connection passed.
2026-08-11 09:10:17,475 [I] --------- Verifying version history table ---------
2026-08-11 09:10:17,475 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklh2cYx/build/asan/bin/postgresql.jar:/tmp/dist-test-tasklh2cYx/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.154.148:44403/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 09:10:17,740 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 09:10:17,740 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklh2cYx/build/asan/bin/postgresql.jar:/tmp/dist-test-tasklh2cYx/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.154.148:44403/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 09:10:18,015 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 09:10:18,016 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-11 09:10:18,016 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklh2cYx/build/asan/bin/postgresql.jar:/tmp/dist-test-tasklh2cYx/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.154.148:44403/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasklh2cYx/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-11 09:10:18.266 UTC [19227] WARNING: there is no transaction in progress
2026-08-11 09:10:18,276 [I] create_dbversion_catalog.sql file imported successfully
2026-08-11 09:10:18,277 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklh2cYx/build/asan/bin/postgresql.jar:/tmp/dist-test-tasklh2cYx/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.154.148:44403/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 09:10:18,553 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-11 09:10:18,553 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 09:10:18,554 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklh2cYx/build/asan/bin/postgresql.jar:/tmp/dist-test-tasklh2cYx/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.154.148:44403/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-11 09:10:18,811 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklh2cYx/build/asan/bin/postgresql.jar:/tmp/dist-test-tasklh2cYx/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.154.148:44403/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-11 09:10:19,066 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklh2cYx/build/asan/bin/postgresql.jar:/tmp/dist-test-tasklh2cYx/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.154.148:44403/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-xt4k.c.gcp-upstream.internal','N') ;"
2026-08-11 09:10:19,316 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklh2cYx/build/asan/bin/postgresql.jar:/tmp/dist-test-tasklh2cYx/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.154.148:44403/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 09:10:19,582 [I] Table x_portal_user does not exist in database ranger
2026-08-11 09:10:19,582 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklh2cYx/build/asan/bin/postgresql.jar:/tmp/dist-test-tasklh2cYx/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.154.148:44403/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 09:10:19,851 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-11 09:10:19,851 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklh2cYx/build/asan/bin/postgresql.jar:/tmp/dist-test-tasklh2cYx/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.154.148:44403/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 09:10:20,124 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklh2cYx/build/asan/bin/postgresql.jar:/tmp/dist-test-tasklh2cYx/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.154.148:44403/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 09:10:20,385 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-11 09:10:20,385 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklh2cYx/build/asan/bin/postgresql.jar:/tmp/dist-test-tasklh2cYx/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.154.148:44403/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasklh2cYx/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-11 09:10:21.450 UTC [19428] WARNING: there is no transaction in progress
2026-08-11 09:10:21.467 UTC [19428] WARNING: there is no transaction in progress
2026-08-11 09:10:21.484 UTC [19428] WARNING: there is no transaction in progress
2026-08-11 09:10:21.500 UTC [19428] WARNING: there is no transaction in progress
2026-08-11 09:10:21.517 UTC [19428] WARNING: there is no transaction in progress
2026-08-11 09:10:21.536 UTC [19428] WARNING: there is no transaction in progress
2026-08-11 09:10:21.598 UTC [19428] WARNING: there is no transaction in progress
2026-08-11 09:10:21.606 UTC [19428] WARNING: there is no transaction in progress
2026-08-11 09:10:21.622 UTC [19428] WARNING: there is no transaction in progress
2026-08-11 09:10:21.632 UTC [19428] WARNING: there is no transaction in progress
2026-08-11 09:10:21.644 UTC [19428] WARNING: there is no transaction in progress
2026-08-11 09:10:21.658 UTC [19428] WARNING: there is no transaction in progress
2026-08-11 09:10:21.669 UTC [19428] WARNING: there is no transaction in progress
2026-08-11 09:10:21.676 UTC [19428] WARNING: there is no transaction in progress
2026-08-11 09:10:21.683 UTC [19428] WARNING: there is no transaction in progress
2026-08-11 09:10:21.690 UTC [19428] WARNING: there is no transaction in progress
2026-08-11 09:10:22,382 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-11 09:10:22,382 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklh2cYx/build/asan/bin/postgresql.jar:/tmp/dist-test-tasklh2cYx/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.154.148:44403/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-11 09:10:22,644 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-11 09:10:22,644 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklh2cYx/build/asan/bin/postgresql.jar:/tmp/dist-test-tasklh2cYx/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.154.148:44403/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 09:10:22,900 [I] Table x_portal_user already exists in database 'ranger'
2026-08-11 09:10:22,900 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklh2cYx/build/asan/bin/postgresql.jar:/tmp/dist-test-tasklh2cYx/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.154.148:44403/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 09:10:23,176 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-11 09:10:23,176 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklh2cYx/build/asan/bin/postgresql.jar:/tmp/dist-test-tasklh2cYx/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.154.148:44403/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 09:10:23,434 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklh2cYx/build/asan/bin/postgresql.jar:/tmp/dist-test-tasklh2cYx/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.154.148:44403/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 09:10:23,686 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklh2cYx/build/asan/bin/postgresql.jar:/tmp/dist-test-tasklh2cYx/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.154.148:44403/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-xt4k.c.gcp-upstream.internal';"
2026-08-11 09:10:23,955 [I] CORE_DB_SCHEMA import status has been updated
2026-08-11 09:10:23,955 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklh2cYx/build/asan/bin/postgresql.jar:/tmp/dist-test-tasklh2cYx/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.154.148:44403/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-11 09:10:24,234 [I] DB_PATCHES have already been applied
I20260811 09:10:24.240458 29290 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ranger-admin:/tmp/dist-test-tasklh2cYx/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-tasklh2cYx/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-tasklh2cYx/build/asan/bin/postgresql.jar:/tmp/dist-test-tasklh2cYx/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260811 09:10:24.240552 29290 mini_ranger.cc:193] Using host: 127.28.154.148
I20260811 09:10:24.260079 29290 mini_ranger.cc:227] Ranger admin URL: http://127.28.154.148:55813
Aug 11, 2026 9:10:24 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 9:10:24 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 9:10:24 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ranger-admin/ews/webapp
Aug 11, 2026 9:10:24 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 11, 2026 9:10:24 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 11, 2026 9:10:25 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ranger-admin/ews/webapp].
Aug 11, 2026 9:10:25 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260811 09:10:25.586742 29290 mini_ranger.cc:160] Time spent starting Ranger: real 9.275s user 0.002s sys 0.058s
Aug 11, 2026 9:10:47 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 11, 2026 9:10:47 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 11, 2026 9:10:47 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 11, 2026 9:10:47 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 11, 2026 9:10:47 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 11, 2026 9:10:47 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 11, 2026 9:10:47 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 11, 2026 9:10:47 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 11, 2026 9:10:47 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 11, 2026 9:10:47 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 11, 2026 9:10:47 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 11, 2026 9:10:47 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 11, 2026 9:10:47 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 11, 2026 9:10:47 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 11, 2026 9:10:47 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 11, 2026 9:10:47 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 11, 2026 9:10:47 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 11, 2026 9:10:48 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260811 09:10:49.459496 29290 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.28.154.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.28.154.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.28.154.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/krb5kdc/rangerkms_127.28.154.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.28.154.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/krb5kdc/rangerkms_127.28.154.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.28.154.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.28.154.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.28.154.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/krb5kdc/HTTP_127.28.154.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.28.154.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/krb5kdc/HTTP_127.28.154.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 11 09:10:49 dist-test-slave-xt4k krb5kdc[19035](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439449, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260811 09:10:49.669718 29290 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260811 09:10:49.698048 29290 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260811 09:10:50.649485 29290 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260811 09:10:50.649613 29290 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ranger-kms
I20260811 09:10:50.649658 29290 mini_ranger_kms.cc:227] Using postgres at 127.28.154.148:44403
2026-08-11 09:10:51,026 [I] DB FLAVOR :POSTGRES
2026-08-11 09:10:51,026 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 09:10:51,026 [I] Checking connection
2026-08-11 09:10:51,026 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklh2cYx/build/asan/bin/postgresql.jar:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.154.148:44403/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-11 09:10:51,327 [I] connection success
2026-08-11 09:10:51,328 [I] --------- Verifying Ranger DB tables ---------
2026-08-11 09:10:51,328 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-11 09:10:51,328 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklh2cYx/build/asan/bin/postgresql.jar:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.154.148:44403/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-11 09:10:51,588 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-11 09:10:51,588 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 09:10:51,589 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-11 09:10:51,589 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasklh2cYx/build/asan/bin/postgresql.jar:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.154.148:44403/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-11 09:10:51,890 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260811 09:10:52.414740 29290 mini_ranger_kms.cc:310] Created kms service
I20260811 09:10:52.583681 29290 mini_ranger_kms.cc:326] Created kudu user
I20260811 09:10:52.632148 29290 mini_ranger_kms.cc:343] Created rangerkms user
Aug 11 09:10:52 dist-test-slave-xt4k krb5kdc[19035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439449, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.28.154.148@KRBTEST.COM
I20260811 09:10:53.054917 29290 mini_ranger_kms.cc:384] Added ranger policy
I20260811 09:10:53.055307 29290 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ranger-kms:/tmp/dist-test-tasklh2cYx/build/asan/bin/postgresql.jar:/tmp/dist-test-tasklh2cYx/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-tasklh2cYx/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260811 09:10:53.055383 29290 mini_ranger_kms.cc:240] Using host: 127.28.154.148
I20260811 09:10:53.076579 29290 mini_ranger_kms.cc:276] Ranger KMS PID: 19749
I20260811 09:10:53.076726 29290 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.28.154.148:50383
Aug 11, 2026 9:10:54 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 9:10:54 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 9:10:54 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 11, 2026 9:10:54 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 11, 2026 9:10:54 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ranger-kms/ews/webapp].
Aug 11, 2026 9:10:54 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 11 09:10:55 dist-test-slave-xt4k krb5kdc[19035](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786439455, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.28.154.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:10:55.799870 29290 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.130s user 0.054s sys 0.353s
I20260811 09:10:55.800118 29290 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260811 09:10:55.800282 29290 mini_ranger_kms.cc:401] 127.28.154.148:50383/kms/v1/keys
Aug 11 09:10:55 dist-test-slave-xt4k krb5kdc[19035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439455, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.28.154.148@KRBTEST.COM for HTTP/127.28.154.148@KRBTEST.COM
Aug 11 09:10:56 dist-test-slave-xt4k krb5kdc[19035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439455, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.28.154.148@KRBTEST.COM for HTTP/127.28.154.148@KRBTEST.COM
Aug 11 09:10:58 dist-test-slave-xt4k krb5kdc[19035](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439458, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.28.154.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.154.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.154.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab.
Entry for principal kudu/127.28.154.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab.
I20260811 09:10:58.757884 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.154.190:40041
--webserver_interface=127.28.154.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab
--principal=kudu/127.28.154.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:35237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.154.190:40041
--ranger_config_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.28.154.148:50383/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:10:59.106683 19828 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:10:59.107004 19828 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:10:59.107111 19828 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:10:59.114527 19828 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:10:59.114657 19828 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:10:59.114709 19828 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:10:59.114789 19828 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:10:59.114845 19828 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:10:59.125294 19828 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:35237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/master-0/wal
--ranger_kms_url=127.28.154.148:50383/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.28.154.190:40041
--ranger_config_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab
--principal=kudu/127.28.154.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.154.190:40041
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.154.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:10:59.126621 19828 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:10:59.128002 19828 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:10:59.141728 19838 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:10:59.141891 19841 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:10:59.141728 19839 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:10:59.142427 19828 server_base.cc:1034] running on GCE node
I20260811 09:10:59.143453 19828 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:10:59.145080 19828 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:10:59.146343 19828 hybrid_clock.cc:648] HybridClock initialized: now 1786439459146289 us; error 53 us; skew 500 ppm
I20260811 09:10:59.146711 19828 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:10:59 dist-test-slave-xt4k krb5kdc[19035](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439459, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:10:59.150041 19828 init.cc:377] Logged in from keytab as kudu/127.28.154.190@KRBTEST.COM (short username kudu)
I20260811 09:10:59.152050 19828 webserver.cc:460] Webserver started at http://127.28.154.190:41823/ using document root <none> and password file <none>
I20260811 09:10:59.152860 19828 fs_manager.cc:359] Metadata directory not provided
I20260811 09:10:59.152983 19828 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:10:59.153319 19828 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 09:10:59 dist-test-slave-xt4k krb5kdc[19035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439459, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.190@KRBTEST.COM for HTTP/127.28.154.148@KRBTEST.COM
I20260811 09:10:59.219170 19828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/master-0/data/instance:
uuid: "81cb27d79a0b47458e95c71620c20cc2"
format_stamp: "Formatted at 2026-08-11 09:10:59 on dist-test-slave-xt4k"
server_key: "239c4165f792a3b681a8a82812807494"
server_key_iv: "b4744585d7519d9c9ac9698fc666a01e"
server_key_version: "kuduclusterkey@0"
I20260811 09:10:59.220297 19828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/master-0/wal/instance:
uuid: "81cb27d79a0b47458e95c71620c20cc2"
format_stamp: "Formatted at 2026-08-11 09:10:59 on dist-test-slave-xt4k"
server_key: "239c4165f792a3b681a8a82812807494"
server_key_iv: "b4744585d7519d9c9ac9698fc666a01e"
server_key_version: "kuduclusterkey@0"
I20260811 09:10:59.227331 19828 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.000s
I20260811 09:10:59.249421 19858 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:10:59.251130 19828 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 09:10:59.251332 19828 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/master-0/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/master-0/wal
uuid: "81cb27d79a0b47458e95c71620c20cc2"
format_stamp: "Formatted at 2026-08-11 09:10:59 on dist-test-slave-xt4k"
server_key: "239c4165f792a3b681a8a82812807494"
server_key_iv: "b4744585d7519d9c9ac9698fc666a01e"
server_key_version: "kuduclusterkey@0"
I20260811 09:10:59.251601 19828 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:10:59.273837 19828 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:10:59.278865 19828 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:10:59.279220 19828 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:10:59.299144 19828 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.190:40041
I20260811 09:10:59.299144 19942 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.190:40041 every 8 connection(s)
I20260811 09:10:59.301070 19828 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
I20260811 09:10:59.305519 19943 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:10:59.311057 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 19828
I20260811 09:10:59.311287 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/master-0/wal/instance
I20260811 09:10:59.312083 29290 external_mini_cluster.cc:1428] Setting key 09b66b4fddb8899cab82820238aa5ebe
I20260811 09:10:59.319499 19943 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81cb27d79a0b47458e95c71620c20cc2: Bootstrap starting.
I20260811 09:10:59.324493 19943 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 81cb27d79a0b47458e95c71620c20cc2: Neither blocks nor log segments found. Creating new log.
I20260811 09:10:59.326308 19943 log.cc:824] T 00000000000000000000000000000000 P 81cb27d79a0b47458e95c71620c20cc2: Log is configured to *not* fsync() on all Append() calls
I20260811 09:10:59.331871 19943 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81cb27d79a0b47458e95c71620c20cc2: No bootstrap required, opened a new log
I20260811 09:10:59.341542 19943 raft_consensus.cc:348] T 00000000000000000000000000000000 P 81cb27d79a0b47458e95c71620c20cc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81cb27d79a0b47458e95c71620c20cc2" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 40041 } }
I20260811 09:10:59.342067 19943 raft_consensus.cc:374] T 00000000000000000000000000000000 P 81cb27d79a0b47458e95c71620c20cc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:10:59.342180 19943 raft_consensus.cc:729] T 00000000000000000000000000000000 P 81cb27d79a0b47458e95c71620c20cc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81cb27d79a0b47458e95c71620c20cc2, State: Initialized, Role: FOLLOWER
I20260811 09:10:59.342967 19943 consensus_queue.cc:260] T 00000000000000000000000000000000 P 81cb27d79a0b47458e95c71620c20cc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81cb27d79a0b47458e95c71620c20cc2" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 40041 } }
I20260811 09:10:59.343243 19943 raft_consensus.cc:388] T 00000000000000000000000000000000 P 81cb27d79a0b47458e95c71620c20cc2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:10:59.343426 19943 raft_consensus.cc:482] T 00000000000000000000000000000000 P 81cb27d79a0b47458e95c71620c20cc2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:10:59.343623 19943 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 81cb27d79a0b47458e95c71620c20cc2 [term 0 FOLLOWER]: Advancing to term 1
Aug 11 09:10:59 dist-test-slave-xt4k krb5kdc[19035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439458, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.154.190@KRBTEST.COM
I20260811 09:10:59.346717 19943 raft_consensus.cc:504] T 00000000000000000000000000000000 P 81cb27d79a0b47458e95c71620c20cc2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81cb27d79a0b47458e95c71620c20cc2" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 40041 } }
I20260811 09:10:59.347328 19943 leader_election.cc:302] T 00000000000000000000000000000000 P 81cb27d79a0b47458e95c71620c20cc2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 81cb27d79a0b47458e95c71620c20cc2; no voters:
I20260811 09:10:59.348081 19943 leader_election.cc:288] T 00000000000000000000000000000000 P 81cb27d79a0b47458e95c71620c20cc2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:10:59.348217 19949 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 81cb27d79a0b47458e95c71620c20cc2 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:10:59.356837 19947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:10:59.314699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35752 (local address 127.28.154.190:40041)
0811 09:10:59.329214 (+ 14515us) server_negotiation.cc:206] Beginning negotiation
0811 09:10:59.329238 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0811 09:10:59.329392 (+ 154us) server_negotiation.cc:407] Connection header received
0811 09:10:59.332022 (+ 2630us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:10:59.332057 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:10:59.332635 (+ 578us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:10:59.333112 (+ 477us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:10:59.334875 (+ 1763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:10:59.337010 (+ 2135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:10:59.339020 (+ 2010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:10:59.340166 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:10:59.340392 (+ 226us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:10:59.345437 (+ 5045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:10:59.345450 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:10:59.345478 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:10:59.345556 (+ 78us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:10:59.350373 (+ 4817us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:10:59.351403 (+ 1030us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:10:59.351414 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:10:59.351425 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:10:59.351651 (+ 226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:10:59.352278 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:10:59.352283 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:10:59.352286 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:10:59.353132 (+ 846us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:10:59.354507 (+ 1375us) server_negotiation.cc:1035] Waiting for connection context
0811 09:10:59.354812 (+ 305us) server_negotiation.cc:299] Negotiation successful
0811 09:10:59.355268 (+ 456us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14197,"thread_start_us":14505,"threads_started":1}
I20260811 09:10:59.360855 19943 sys_catalog.cc:564] T 00000000000000000000000000000000 P 81cb27d79a0b47458e95c71620c20cc2 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:10:59.360981 19949 raft_consensus.cc:686] T 00000000000000000000000000000000 P 81cb27d79a0b47458e95c71620c20cc2 [term 1 LEADER]: Becoming Leader. State: Replica: 81cb27d79a0b47458e95c71620c20cc2, State: Running, Role: LEADER
I20260811 09:10:59.361575 19949 consensus_queue.cc:237] T 00000000000000000000000000000000 P 81cb27d79a0b47458e95c71620c20cc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81cb27d79a0b47458e95c71620c20cc2" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 40041 } }
I20260811 09:10:59.364602 19951 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81cb27d79a0b47458e95c71620c20cc2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "81cb27d79a0b47458e95c71620c20cc2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81cb27d79a0b47458e95c71620c20cc2" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 40041 } } }
I20260811 09:10:59.365038 19951 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81cb27d79a0b47458e95c71620c20cc2 [sys.catalog]: This master's current role is: LEADER
I20260811 09:10:59.365552 19952 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81cb27d79a0b47458e95c71620c20cc2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 81cb27d79a0b47458e95c71620c20cc2. Latest consensus state: current_term: 1 leader_uuid: "81cb27d79a0b47458e95c71620c20cc2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81cb27d79a0b47458e95c71620c20cc2" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 40041 } } }
I20260811 09:10:59.365890 19952 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81cb27d79a0b47458e95c71620c20cc2 [sys.catalog]: This master's current role is: LEADER
I20260811 09:10:59.366094 19963 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:10:59.373075 19963 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:10:59.384519 19963 catalog_manager.cc:1347] Generated new cluster ID: 5d7a1fb5751a4a5883c30e97378614ce
I20260811 09:10:59.384660 19963 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:10:59.408305 19963 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:10:59.408553 19943 ranger_client.cc:318] Using new properties file: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 09:10:59.410337 19963 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:10:59.423818 19963 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 81cb27d79a0b47458e95c71620c20cc2: Generated new TSK 0
I20260811 09:10:59.432034 19963 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.28.154.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.154.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.154.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.129.keytab.
Entry for principal kudu/127.28.154.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.129.keytab.
I20260811 09:11:00.314304 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.28.154.148:50383/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.154.129:0
--local_ip_for_outbound_sockets=127.28.154.129
--webserver_interface=127.28.154.129
--webserver_port=0
--tserver_master_addrs=127.28.154.190:40041
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.129.keytab
--principal=kudu/127.28.154.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:35237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 09:11:00 dist-test-slave-xt4k krb5kdc[19035](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786439460, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260811 09:11:00.774127 20025 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:11:00.774549 20025 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:11:00.774823 20025 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:11:00.787861 20025 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:11:00.788051 20025 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:11:00.788306 20025 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.154.129
I20260811 09:11:00.805296 20025 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:35237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.28.154.148:50383/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.129.keytab
--principal=kudu/127.28.154.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.154.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.154.129
--webserver_port=0
--tserver_master_addrs=127.28.154.190:40041
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.154.129
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:11:00.807164 20025 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:11:00.808648 20025 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:11:00.826607 20042 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:11:00.827178 20045 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:11:00.827346 20039 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:11:00.829021 20025 server_base.cc:1034] running on GCE node
I20260811 09:11:00.829808 20025 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:11:00.831065 20025 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:11:00.832365 20025 hybrid_clock.cc:648] HybridClock initialized: now 1786439460832255 us; error 76 us; skew 500 ppm
I20260811 09:11:00.832790 20025 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:11:00 dist-test-slave-xt4k krb5kdc[19035](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439460, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:11:00.836855 20025 init.cc:377] Logged in from keytab as kudu/127.28.154.129@KRBTEST.COM (short username kudu)
I20260811 09:11:00.838397 20025 webserver.cc:460] Webserver started at http://127.28.154.129:39127/ using document root <none> and password file <none>
I20260811 09:11:00.839104 20025 fs_manager.cc:359] Metadata directory not provided
I20260811 09:11:00.839226 20025 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:11:00.839545 20025 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 09:11:00 dist-test-slave-xt4k krb5kdc[19035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439460, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.129@KRBTEST.COM for HTTP/127.28.154.148@KRBTEST.COM
I20260811 09:11:00.859467 20025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ts-0/data/instance:
uuid: "028deeec53e24908a86608129742f470"
format_stamp: "Formatted at 2026-08-11 09:11:00 on dist-test-slave-xt4k"
server_key: "e42a6c820f84f525a1339ccf94c38a0a"
server_key_iv: "be15b752314cc2485bbc938d01380063"
server_key_version: "kuduclusterkey@0"
I20260811 09:11:00.860705 20025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ts-0/wal/instance:
uuid: "028deeec53e24908a86608129742f470"
format_stamp: "Formatted at 2026-08-11 09:11:00 on dist-test-slave-xt4k"
server_key: "e42a6c820f84f525a1339ccf94c38a0a"
server_key_iv: "be15b752314cc2485bbc938d01380063"
server_key_version: "kuduclusterkey@0"
I20260811 09:11:00.880662 20025 fs_manager.cc:691] Time spent creating directory manager: real 0.019s user 0.014s sys 0.005s
I20260811 09:11:00.906107 20059 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:11:00.908432 20025 fs_manager.cc:725] Time spent opening block manager: real 0.009s user 0.005s sys 0.003s
I20260811 09:11:00.908697 20025 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ts-0/wal
uuid: "028deeec53e24908a86608129742f470"
format_stamp: "Formatted at 2026-08-11 09:11:00 on dist-test-slave-xt4k"
server_key: "e42a6c820f84f525a1339ccf94c38a0a"
server_key_iv: "be15b752314cc2485bbc938d01380063"
server_key_version: "kuduclusterkey@0"
I20260811 09:11:00.909004 20025 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:11:00.962558 20025 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:11:00.972774 20025 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:11:00.973409 20025 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:11:00.984859 20025 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:11:01.006119 20025 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:11:01.009084 20025 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.003s user 0.001s sys 0.000s
I20260811 09:11:01.009302 20025 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:11:01.009373 20025 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 09:11:01 dist-test-slave-xt4k krb5kdc[19035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439460, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.190@KRBTEST.COM for HTTP/127.28.154.148@KRBTEST.COM
I20260811 09:11:01.344556 20025 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.129:33165
I20260811 09:11:01.344597 20276 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.129:33165 every 8 connection(s)
I20260811 09:11:01.346618 20025 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ts-0/data/info.pb
I20260811 09:11:01.357107 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 20025
I20260811 09:11:01.357347 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ts-0/wal/instance
I20260811 09:11:01.358160 29290 external_mini_cluster.cc:1428] Setting key ce0046a825aedf0f8b19b6e5bee9a020
Aug 11 09:11:01 dist-test-slave-xt4k krb5kdc[19035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439460, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.129@KRBTEST.COM for kudu/127.28.154.190@KRBTEST.COM
I20260811 09:11:01.377887 20280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:01.351161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.129:51337 (local address 127.28.154.190:40041)
0811 09:11:01.353251 (+ 2090us) server_negotiation.cc:206] Beginning negotiation
0811 09:11:01.353258 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:11:01.353284 (+ 26us) server_negotiation.cc:407] Connection header received
0811 09:11:01.354940 (+ 1656us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:11:01.354946 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:11:01.355034 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:11:01.355159 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:11:01.357575 (+ 2416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:01.358794 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:01.361783 (+ 2989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:01.362548 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:01.362672 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:01.367548 (+ 4876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:11:01.367556 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:11:01.367562 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:11:01.367608 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:11:01.371114 (+ 3506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:11:01.372125 (+ 1011us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:11:01.372132 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:11:01.372135 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:11:01.372305 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:11:01.372863 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:11:01.372868 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:11:01.372871 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:11:01.373917 (+ 1046us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:11:01.374070 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0811 09:11:01.377233 (+ 3163us) server_negotiation.cc:299] Negotiation successful
0811 09:11:01.377492 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1959,"thread_start_us":1793,"threads_started":1}
I20260811 09:11:01.384928 20279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:01.350640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:40041 (local address 127.28.154.129:51337)
0811 09:11:01.351609 (+ 969us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:01.351666 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 09:11:01.354565 (+ 2899us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:11:01.355467 (+ 902us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:11:01.355481 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:11:01.356259 (+ 778us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:11:01.357318 (+ 1059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:11:01.357347 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:11:01.359021 (+ 1674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:11:01.359027 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:11:01.361586 (+ 2559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:11:01.361606 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:11:01.363031 (+ 1425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:11:01.363039 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:11:01.363366 (+ 327us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:01.364171 (+ 805us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:11:01.364202 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:11:01.367218 (+ 3016us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:11:01.371343 (+ 4125us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:11:01.371354 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:11:01.371375 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:11:01.371972 (+ 597us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:11:01.372458 (+ 486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:11:01.372462 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:11:01.372466 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:11:01.372730 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:11:01.374150 (+ 1420us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:11:01.374159 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:11:01.374683 (+ 524us) client_negotiation.cc:769] Sending connection context
0811 09:11:01.375046 (+ 363us) client_negotiation.cc:240] Negotiation successful
0811 09:11:01.375707 (+ 661us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":637,"thread_start_us":599,"threads_started":1}
I20260811 09:11:01.387876 20277 heartbeater.cc:347] Connected to a master server at 127.28.154.190:40041
I20260811 09:11:01.388271 20277 heartbeater.cc:464] Registering TS with master...
I20260811 09:11:01.389390 20277 heartbeater.cc:511] Master 127.28.154.190:40041 requested a full tablet report, sending...
I20260811 09:11:01.392580 19885 ts_manager.cc:194] Registered new tserver with Master: 028deeec53e24908a86608129742f470 (127.28.154.129:33165)
I20260811 09:11:01.394975 19885 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.154.129@KRBTEST.COM'} at 127.28.154.129:51337
WARNING: no policy specified for kudu/127.28.154.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.154.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.154.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.130.keytab.
Entry for principal kudu/127.28.154.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.130.keytab.
I20260811 09:11:01.432225 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.28.154.148:50383/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.154.130:0
--local_ip_for_outbound_sockets=127.28.154.130
--webserver_interface=127.28.154.130
--webserver_port=0
--tserver_master_addrs=127.28.154.190:40041
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.130.keytab
--principal=kudu/127.28.154.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:35237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 09:11:01 dist-test-slave-xt4k krb5kdc[19035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439460, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.190@KRBTEST.COM for HTTP/127.28.154.148@KRBTEST.COM
W20260811 09:11:01.967685 20283 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:11:01.968024 20283 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:11:01.968211 20283 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:11:01.976181 20283 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:11:01.976325 20283 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:11:01.976521 20283 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.154.130
I20260811 09:11:01.987812 20283 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:35237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.28.154.148:50383/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.130.keytab
--principal=kudu/127.28.154.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.154.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.154.130
--webserver_port=0
--tserver_master_addrs=127.28.154.190:40041
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.154.130
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:11:01.989327 20283 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:11:01.990702 20283 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:11:02.008670 20303 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:11:02.009858 20304 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:11:02.018563 20309 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:11:02.019105 20283 server_base.cc:1034] running on GCE node
I20260811 09:11:02.019886 20283 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:11:02.021214 20283 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:11:02.022557 20283 hybrid_clock.cc:648] HybridClock initialized: now 1786439462022462 us; error 87 us; skew 500 ppm
I20260811 09:11:02.022920 20283 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:11:02 dist-test-slave-xt4k krb5kdc[19035](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439462, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:11:02.026737 20283 init.cc:377] Logged in from keytab as kudu/127.28.154.130@KRBTEST.COM (short username kudu)
I20260811 09:11:02.030647 20283 webserver.cc:460] Webserver started at http://127.28.154.130:35631/ using document root <none> and password file <none>
I20260811 09:11:02.031476 20283 fs_manager.cc:359] Metadata directory not provided
I20260811 09:11:02.031610 20283 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:11:02.031944 20283 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 09:11:02 dist-test-slave-xt4k krb5kdc[19035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439462, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.130@KRBTEST.COM for HTTP/127.28.154.148@KRBTEST.COM
I20260811 09:11:02.050757 20283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ts-1/data/instance:
uuid: "99dfb45b51a54908bc7312a5675b6899"
format_stamp: "Formatted at 2026-08-11 09:11:02 on dist-test-slave-xt4k"
server_key: "3e07135aa18402453dbfcdc57f865f60"
server_key_iv: "555121fc3c3a45fb01a5dbd45f423891"
server_key_version: "kuduclusterkey@0"
I20260811 09:11:02.052124 20283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ts-1/wal/instance:
uuid: "99dfb45b51a54908bc7312a5675b6899"
format_stamp: "Formatted at 2026-08-11 09:11:02 on dist-test-slave-xt4k"
server_key: "3e07135aa18402453dbfcdc57f865f60"
server_key_iv: "555121fc3c3a45fb01a5dbd45f423891"
server_key_version: "kuduclusterkey@0"
I20260811 09:11:02.061650 20283 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.004s sys 0.003s
I20260811 09:11:02.080516 20322 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:11:02.082522 20283 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 09:11:02.082756 20283 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ts-1/wal
uuid: "99dfb45b51a54908bc7312a5675b6899"
format_stamp: "Formatted at 2026-08-11 09:11:02 on dist-test-slave-xt4k"
server_key: "3e07135aa18402453dbfcdc57f865f60"
server_key_iv: "555121fc3c3a45fb01a5dbd45f423891"
server_key_version: "kuduclusterkey@0"
I20260811 09:11:02.083040 20283 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:11:02.108078 20283 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:11:02.114012 20283 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:11:02.114403 20283 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:11:02.115922 20283 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:11:02.117956 20283 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:11:02.118068 20283 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:11:02.118257 20283 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:11:02.118330 20283 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:11:02.181674 20283 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.130:40537
I20260811 09:11:02.181710 20510 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.130:40537 every 8 connection(s)
I20260811 09:11:02.183637 20283 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ts-1/data/info.pb
I20260811 09:11:02.185359 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 20283
I20260811 09:11:02.185534 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ts-1/wal/instance
I20260811 09:11:02.186064 29290 external_mini_cluster.cc:1428] Setting key 142d39708bae286f1795e7ef55ac754a
Aug 11 09:11:02 dist-test-slave-xt4k krb5kdc[19035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439462, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.130@KRBTEST.COM for kudu/127.28.154.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.28.154.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.154.131@KRBTEST.COM" created.
I20260811 09:11:02.243009 20520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:02.200490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:40041 (local address 127.28.154.130:33651)
0811 09:11:02.209075 (+ 8585us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:02.209151 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0811 09:11:02.211783 (+ 2632us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:11:02.216886 (+ 5103us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:11:02.216902 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:11:02.217634 (+ 732us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:11:02.218666 (+ 1032us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:11:02.218700 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:11:02.220410 (+ 1710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:11:02.220415 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:11:02.222579 (+ 2164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:11:02.222613 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:11:02.224009 (+ 1396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:11:02.224021 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:11:02.224384 (+ 363us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:02.225234 (+ 850us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:11:02.225266 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:11:02.228207 (+ 2941us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:11:02.232867 (+ 4660us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:11:02.232897 (+ 30us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:11:02.232921 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:11:02.233810 (+ 889us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:11:02.234366 (+ 556us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:11:02.234371 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:11:02.234374 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:11:02.234713 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:11:02.235470 (+ 757us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:11:02.235480 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:11:02.236047 (+ 567us) client_negotiation.cc:769] Sending connection context
0811 09:11:02.240702 (+ 4655us) client_negotiation.cc:240] Negotiation successful
0811 09:11:02.241395 (+ 693us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8172,"thread_start_us":8275,"threads_started":1}
I20260811 09:11:02.244240 20519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:02.199767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.130:33651 (local address 127.28.154.190:40041)
0811 09:11:02.215057 (+ 15290us) server_negotiation.cc:206] Beginning negotiation
0811 09:11:02.215064 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:11:02.215091 (+ 27us) server_negotiation.cc:407] Connection header received
0811 09:11:02.215165 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:11:02.215171 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:11:02.215259 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:11:02.215403 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:11:02.218953 (+ 3550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:02.220147 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:02.222858 (+ 2711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:02.223489 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:02.224883 (+ 1394us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:02.228593 (+ 3710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:11:02.228601 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:11:02.228605 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:11:02.228653 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:11:02.232642 (+ 3989us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:11:02.233996 (+ 1354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:11:02.234003 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:11:02.234007 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:11:02.234194 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:11:02.234856 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:11:02.234863 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:11:02.234867 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:11:02.235298 (+ 431us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:11:02.235461 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0811 09:11:02.243532 (+ 8071us) server_negotiation.cc:299] Negotiation successful
0811 09:11:02.243849 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15123,"thread_start_us":15742,"threads_started":1}
I20260811 09:11:02.246513 20511 heartbeater.cc:347] Connected to a master server at 127.28.154.190:40041
I20260811 09:11:02.246883 20511 heartbeater.cc:464] Registering TS with master...
I20260811 09:11:02.247851 20511 heartbeater.cc:511] Master 127.28.154.190:40041 requested a full tablet report, sending...
I20260811 09:11:02.249910 19885 ts_manager.cc:194] Registered new tserver with Master: 99dfb45b51a54908bc7312a5675b6899 (127.28.154.130:40537)
I20260811 09:11:02.251073 19885 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.154.130@KRBTEST.COM'} at 127.28.154.130:33651
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.154.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.131.keytab.
Entry for principal kudu/127.28.154.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.131.keytab.
I20260811 09:11:02.276072 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.28.154.148:50383/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.154.131:0
--local_ip_for_outbound_sockets=127.28.154.131
--webserver_interface=127.28.154.131
--webserver_port=0
--tserver_master_addrs=127.28.154.190:40041
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.131.keytab
--principal=kudu/127.28.154.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:35237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 09:11:02.399314 20277 heartbeater.cc:503] Master 127.28.154.190:40041 was elected leader, sending a full tablet report...
W20260811 09:11:02.681600 20526 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:11:02.681910 20526 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:11:02.682056 20526 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:11:02.690107 20526 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:11:02.690249 20526 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:11:02.690436 20526 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.154.131
I20260811 09:11:02.702221 20526 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:35237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.28.154.148:50383/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.131.keytab
--principal=kudu/127.28.154.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.154.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.154.131
--webserver_port=0
--tserver_master_addrs=127.28.154.190:40041
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.154.131
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:11:02.703667 20526 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:11:02.705050 20526 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:11:02.718320 20531 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:11:02.718348 20536 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:11:02.718401 20532 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:11:02.719651 20526 server_base.cc:1034] running on GCE node
I20260811 09:11:02.720252 20526 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:11:02.721314 20526 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:11:02.722563 20526 hybrid_clock.cc:648] HybridClock initialized: now 1786439462722503 us; error 40 us; skew 500 ppm
I20260811 09:11:02.722937 20526 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:11:02 dist-test-slave-xt4k krb5kdc[19035](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439462, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:11:02.726420 20526 init.cc:377] Logged in from keytab as kudu/127.28.154.131@KRBTEST.COM (short username kudu)
I20260811 09:11:02.728035 20526 webserver.cc:460] Webserver started at http://127.28.154.131:43717/ using document root <none> and password file <none>
I20260811 09:11:02.728826 20526 fs_manager.cc:359] Metadata directory not provided
I20260811 09:11:02.728989 20526 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:11:02.729312 20526 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 09:11:02 dist-test-slave-xt4k krb5kdc[19035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439462, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.131@KRBTEST.COM for HTTP/127.28.154.148@KRBTEST.COM
I20260811 09:11:02.744372 20526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ts-2/data/instance:
uuid: "2805ee604f62472f8e512ef3e1153cda"
format_stamp: "Formatted at 2026-08-11 09:11:02 on dist-test-slave-xt4k"
server_key: "025f4c88ae6effe63c329d5aa2d6fecb"
server_key_iv: "74e7664078104c09a9ac689393d81b70"
server_key_version: "kuduclusterkey@0"
I20260811 09:11:02.745534 20526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ts-2/wal/instance:
uuid: "2805ee604f62472f8e512ef3e1153cda"
format_stamp: "Formatted at 2026-08-11 09:11:02 on dist-test-slave-xt4k"
server_key: "025f4c88ae6effe63c329d5aa2d6fecb"
server_key_iv: "74e7664078104c09a9ac689393d81b70"
server_key_version: "kuduclusterkey@0"
I20260811 09:11:02.751688 20526 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260811 09:11:02.768393 20548 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:11:02.770133 20526 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 09:11:02.770323 20526 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ts-2/wal
uuid: "2805ee604f62472f8e512ef3e1153cda"
format_stamp: "Formatted at 2026-08-11 09:11:02 on dist-test-slave-xt4k"
server_key: "025f4c88ae6effe63c329d5aa2d6fecb"
server_key_iv: "74e7664078104c09a9ac689393d81b70"
server_key_version: "kuduclusterkey@0"
I20260811 09:11:02.770582 20526 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:11:02.806427 20526 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:11:02.810724 20526 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:11:02.811069 20526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:11:02.812206 20526 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:11:02.813766 20526 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:11:02.813848 20526 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:11:02.813980 20526 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:11:02.814038 20526 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:11:02.860811 20526 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.131:45265
I20260811 09:11:02.860872 20758 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.131:45265 every 8 connection(s)
I20260811 09:11:02.862339 20526 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ts-2/data/info.pb
I20260811 09:11:02.872500 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 20526
I20260811 09:11:02.872697 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ts-2/wal/instance
I20260811 09:11:02.873345 29290 external_mini_cluster.cc:1428] Setting key 287566a28444d5cc1618b77088fcd4e1
Aug 11 09:11:02 dist-test-slave-xt4k krb5kdc[19035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439462, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.131@KRBTEST.COM for kudu/127.28.154.190@KRBTEST.COM
I20260811 09:11:02.885648 20763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:02.865423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.131:38239 (local address 127.28.154.190:40041)
0811 09:11:02.865971 (+ 548us) server_negotiation.cc:206] Beginning negotiation
0811 09:11:02.865983 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 09:11:02.867147 (+ 1164us) server_negotiation.cc:407] Connection header received
0811 09:11:02.869076 (+ 1929us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:11:02.869081 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:11:02.869185 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:11:02.869287 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:11:02.870800 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:02.871571 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:02.873889 (+ 2318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:02.874650 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:02.874785 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:02.879229 (+ 4444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:11:02.879235 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:11:02.879239 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:11:02.879277 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:11:02.882051 (+ 2774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:11:02.883088 (+ 1037us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:11:02.883093 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:11:02.883095 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:11:02.883228 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:11:02.883792 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:11:02.883796 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:11:02.883798 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:11:02.884098 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:11:02.884216 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 09:11:02.885199 (+ 983us) server_negotiation.cc:299] Negotiation successful
0811 09:11:02.885380 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":445,"thread_start_us":321,"threads_started":1}
I20260811 09:11:02.887276 20764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:02.866141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:40041 (local address 127.28.154.131:38239)
0811 09:11:02.866965 (+ 824us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:02.867029 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0811 09:11:02.868798 (+ 1769us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:11:02.869540 (+ 742us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:11:02.869549 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:11:02.870017 (+ 468us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:11:02.870616 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:11:02.870633 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:11:02.871734 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:11:02.871738 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:11:02.873688 (+ 1950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:11:02.873705 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:11:02.874918 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:11:02.874927 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:11:02.875293 (+ 366us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:02.876031 (+ 738us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:11:02.876064 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:11:02.878995 (+ 2931us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:11:02.882204 (+ 3209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:11:02.882215 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:11:02.882233 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:11:02.882965 (+ 732us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:11:02.883331 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:11:02.883336 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:11:02.883340 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:11:02.883667 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:11:02.884227 (+ 560us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:11:02.884242 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:11:02.884770 (+ 528us) client_negotiation.cc:769] Sending connection context
0811 09:11:02.885112 (+ 342us) client_negotiation.cc:240] Negotiation successful
0811 09:11:02.885693 (+ 581us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":576,"thread_start_us":348,"threads_started":1}
I20260811 09:11:02.889007 20759 heartbeater.cc:347] Connected to a master server at 127.28.154.190:40041
I20260811 09:11:02.889343 20759 heartbeater.cc:464] Registering TS with master...
I20260811 09:11:02.890192 20759 heartbeater.cc:511] Master 127.28.154.190:40041 requested a full tablet report, sending...
I20260811 09:11:02.891611 19885 ts_manager.cc:194] Registered new tserver with Master: 2805ee604f62472f8e512ef3e1153cda (127.28.154.131:45265)
I20260811 09:11:02.892493 19885 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.154.131@KRBTEST.COM'} at 127.28.154.131:38239
I20260811 09:11:02.902520 29290 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 09:11:02.905051 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 20025
Aug 11 09:11:02 dist-test-slave-xt4k krb5kdc[19035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439458, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.154.129@KRBTEST.COM
I20260811 09:11:02.924371 20768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:02.906533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47902 (local address 127.28.154.129:33165)
0811 09:11:02.907209 (+ 676us) server_negotiation.cc:206] Beginning negotiation
0811 09:11:02.907216 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:11:02.907243 (+ 27us) server_negotiation.cc:407] Connection header received
0811 09:11:02.907300 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:11:02.907307 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:11:02.907557 (+ 250us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:11:02.907725 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:11:02.909517 (+ 1792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:02.910576 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:02.912027 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:02.912870 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:02.912995 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:02.918347 (+ 5352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:11:02.918359 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:11:02.918377 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:11:02.918423 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:11:02.921344 (+ 2921us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:11:02.922250 (+ 906us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:11:02.922259 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:11:02.922261 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:11:02.922440 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:11:02.922829 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:11:02.922832 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:11:02.922834 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:11:02.923194 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:11:02.923431 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0811 09:11:02.923790 (+ 359us) server_negotiation.cc:299] Negotiation successful
0811 09:11:02.924028 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":513,"thread_start_us":347,"threads_started":1}
I20260811 09:11:02.925868 20269 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:11:03.010974 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 20025
I20260811 09:11:03.046314 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 20283
Aug 11 09:11:03 dist-test-slave-xt4k krb5kdc[19035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439458, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.154.130@KRBTEST.COM
I20260811 09:11:03.065793 20771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:03.047246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48536 (local address 127.28.154.130:40537)
0811 09:11:03.047765 (+ 519us) server_negotiation.cc:206] Beginning negotiation
0811 09:11:03.047773 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:11:03.047802 (+ 29us) server_negotiation.cc:407] Connection header received
0811 09:11:03.047865 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:11:03.047872 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:11:03.048089 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:11:03.048232 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:11:03.049856 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:03.050891 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:03.052888 (+ 1997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:03.053508 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:03.053619 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:03.058818 (+ 5199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:11:03.058833 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:11:03.058854 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:11:03.058916 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:11:03.062610 (+ 3694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:11:03.063438 (+ 828us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:11:03.063448 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:11:03.063451 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:11:03.063637 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:11:03.063995 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:11:03.064000 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:11:03.064004 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:11:03.064494 (+ 490us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:11:03.064776 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0811 09:11:03.065108 (+ 332us) server_negotiation.cc:299] Negotiation successful
0811 09:11:03.065387 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":364,"thread_start_us":267,"threads_started":1}
I20260811 09:11:03.067253 20503 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:11:03.150743 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 20283
I20260811 09:11:03.174374 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 20526
Aug 11 09:11:03 dist-test-slave-xt4k krb5kdc[19035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439458, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.154.131@KRBTEST.COM
I20260811 09:11:03.192044 20774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:03.175440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37532 (local address 127.28.154.131:45265)
0811 09:11:03.176194 (+ 754us) server_negotiation.cc:206] Beginning negotiation
0811 09:11:03.176202 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:11:03.176233 (+ 31us) server_negotiation.cc:407] Connection header received
0811 09:11:03.176302 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:11:03.176308 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:11:03.176607 (+ 299us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:11:03.176752 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:11:03.178144 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:03.179340 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:03.180717 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:03.181281 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:03.181381 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:03.185922 (+ 4541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:11:03.185943 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:11:03.185960 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:11:03.186000 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:11:03.189084 (+ 3084us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:11:03.189839 (+ 755us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:11:03.189847 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:11:03.189850 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:11:03.190022 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:11:03.190458 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:11:03.190463 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:11:03.190465 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:11:03.190901 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:11:03.191167 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0811 09:11:03.191512 (+ 345us) server_negotiation.cc:299] Negotiation successful
0811 09:11:03.191743 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":604,"thread_start_us":443,"threads_started":1}
I20260811 09:11:03.193452 20753 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:11:03.278544 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 20526
I20260811 09:11:03.303318 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 19828
I20260811 09:11:03.304744 19938 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:11:03.412470 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 19828
I20260811 09:11:03.443666 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.154.190:40041
--webserver_interface=127.28.154.190
--webserver_port=41823
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab
--principal=kudu/127.28.154.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:35237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.154.190:40041
--ranger_config_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.28.154.148:50383/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.28.154.148:50383/kms with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:11:03.776537 20777 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:11:03.776830 20777 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:11:03.776916 20777 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:11:03.784122 20777 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:11:03.784235 20777 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:11:03.784282 20777 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:11:03.784358 20777 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:11:03.784407 20777 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:11:03.794685 20777 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:35237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.28.154.148:50383/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.28.154.190:40041
--ranger_config_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab
--principal=kudu/127.28.154.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.154.190:40041
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.154.190
--webserver_port=41823
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:11:03.795951 20777 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:11:03.797360 20777 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:11:03.809481 20790 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:11:03.809551 20792 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:11:03.809808 20795 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:11:03.810887 20777 server_base.cc:1034] running on GCE node
I20260811 09:11:03.811662 20777 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:11:03.813155 20777 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:11:03.814366 20777 hybrid_clock.cc:648] HybridClock initialized: now 1786439463814319 us; error 45 us; skew 500 ppm
I20260811 09:11:03.814723 20777 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:11:03 dist-test-slave-xt4k krb5kdc[19035](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439463, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:11:03.817903 20777 init.cc:377] Logged in from keytab as kudu/127.28.154.190@KRBTEST.COM (short username kudu)
I20260811 09:11:03.819382 20777 webserver.cc:460] Webserver started at http://127.28.154.190:41823/ using document root <none> and password file <none>
I20260811 09:11:03.820053 20777 fs_manager.cc:359] Metadata directory not provided
I20260811 09:11:03.820181 20777 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 09:11:03 dist-test-slave-xt4k krb5kdc[19035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439463, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.190@KRBTEST.COM for HTTP/127.28.154.148@KRBTEST.COM
I20260811 09:11:03.866514 20777 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.004s sys 0.000s
I20260811 09:11:03.869767 20809 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:11:03.871158 20777 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260811 09:11:03.871330 20777 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/master-0/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/master-0/wal
uuid: "81cb27d79a0b47458e95c71620c20cc2"
format_stamp: "Formatted at 2026-08-11 09:10:59 on dist-test-slave-xt4k"
server_key: "239c4165f792a3b681a8a82812807494"
server_key_iv: "b4744585d7519d9c9ac9698fc666a01e"
server_key_version: "kuduclusterkey@0"
I20260811 09:11:03.872282 20777 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:11:03.889994 20777 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:11:03.894375 20777 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:11:03.894671 20777 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:11:03.912709 20777 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.190:40041
I20260811 09:11:03.912716 20900 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.190:40041 every 8 connection(s)
I20260811 09:11:03.914469 20777 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
I20260811 09:11:03.921576 20901 sys_catalog.cc:263] Verifying existing consensus state
I20260811 09:11:03.922081 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 20777
I20260811 09:11:03.922936 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.28.154.148:50383/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.154.129:33165
--local_ip_for_outbound_sockets=127.28.154.129
--tserver_master_addrs=127.28.154.190:40041
--webserver_port=39127
--webserver_interface=127.28.154.129
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.129.keytab
--principal=kudu/127.28.154.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:35237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.28.154.148:50383/kms with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 09:11:03.925132 20901 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81cb27d79a0b47458e95c71620c20cc2: Bootstrap starting.
I20260811 09:11:03.946955 20901 log.cc:824] T 00000000000000000000000000000000 P 81cb27d79a0b47458e95c71620c20cc2: Log is configured to *not* fsync() on all Append() calls
I20260811 09:11:03.959054 20901 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81cb27d79a0b47458e95c71620c20cc2: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 09:11:03.959708 20901 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81cb27d79a0b47458e95c71620c20cc2: Bootstrap complete.
I20260811 09:11:03.970527 20901 raft_consensus.cc:348] T 00000000000000000000000000000000 P 81cb27d79a0b47458e95c71620c20cc2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81cb27d79a0b47458e95c71620c20cc2" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 40041 } }
I20260811 09:11:03.971417 20901 raft_consensus.cc:729] T 00000000000000000000000000000000 P 81cb27d79a0b47458e95c71620c20cc2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81cb27d79a0b47458e95c71620c20cc2, State: Initialized, Role: FOLLOWER
I20260811 09:11:03.972038 20901 consensus_queue.cc:260] T 00000000000000000000000000000000 P 81cb27d79a0b47458e95c71620c20cc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81cb27d79a0b47458e95c71620c20cc2" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 40041 } }
I20260811 09:11:03.972257 20901 raft_consensus.cc:388] T 00000000000000000000000000000000 P 81cb27d79a0b47458e95c71620c20cc2 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:11:03.972388 20901 raft_consensus.cc:482] T 00000000000000000000000000000000 P 81cb27d79a0b47458e95c71620c20cc2 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:11:03.972555 20901 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 81cb27d79a0b47458e95c71620c20cc2 [term 1 FOLLOWER]: Advancing to term 2
I20260811 09:11:03.976457 20901 raft_consensus.cc:504] T 00000000000000000000000000000000 P 81cb27d79a0b47458e95c71620c20cc2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81cb27d79a0b47458e95c71620c20cc2" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 40041 } }
I20260811 09:11:03.976958 20901 leader_election.cc:302] T 00000000000000000000000000000000 P 81cb27d79a0b47458e95c71620c20cc2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 81cb27d79a0b47458e95c71620c20cc2; no voters:
I20260811 09:11:03.977557 20901 leader_election.cc:288] T 00000000000000000000000000000000 P 81cb27d79a0b47458e95c71620c20cc2 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 09:11:03.977739 20908 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 81cb27d79a0b47458e95c71620c20cc2 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 09:11:03.978737 20908 raft_consensus.cc:686] T 00000000000000000000000000000000 P 81cb27d79a0b47458e95c71620c20cc2 [term 2 LEADER]: Becoming Leader. State: Replica: 81cb27d79a0b47458e95c71620c20cc2, State: Running, Role: LEADER
I20260811 09:11:03.979518 20901 sys_catalog.cc:564] T 00000000000000000000000000000000 P 81cb27d79a0b47458e95c71620c20cc2 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:11:03.979378 20908 consensus_queue.cc:237] T 00000000000000000000000000000000 P 81cb27d79a0b47458e95c71620c20cc2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81cb27d79a0b47458e95c71620c20cc2" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 40041 } }
I20260811 09:11:03.985436 20901 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 09:11:03.985386 20909 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81cb27d79a0b47458e95c71620c20cc2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "81cb27d79a0b47458e95c71620c20cc2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81cb27d79a0b47458e95c71620c20cc2" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 40041 } } }
I20260811 09:11:03.985795 20909 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81cb27d79a0b47458e95c71620c20cc2 [sys.catalog]: This master's current role is: LEADER
I20260811 09:11:03.986292 20910 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81cb27d79a0b47458e95c71620c20cc2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 81cb27d79a0b47458e95c71620c20cc2. Latest consensus state: current_term: 2 leader_uuid: "81cb27d79a0b47458e95c71620c20cc2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81cb27d79a0b47458e95c71620c20cc2" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 40041 } } }
I20260811 09:11:03.986594 20910 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81cb27d79a0b47458e95c71620c20cc2 [sys.catalog]: This master's current role is: LEADER
I20260811 09:11:04.015174 20931 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:11:04.023460 20931 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:11:04.025259 20931 catalog_manager.cc:1259] Loaded cluster ID: 5d7a1fb5751a4a5883c30e97378614ce
I20260811 09:11:04.025409 20931 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:11:04.029945 20931 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:11:04.031699 20931 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 81cb27d79a0b47458e95c71620c20cc2: Loaded TSK: 0
I20260811 09:11:04.033370 20931 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 09:11:04.367486 20904 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:11:04.368010 20904 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:11:04.368240 20904 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:11:04.379722 20904 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:11:04.379990 20904 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:11:04.380237 20904 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.154.129
I20260811 09:11:04.393579 20904 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:35237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.28.154.148:50383/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.129.keytab
--principal=kudu/127.28.154.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.154.129:33165
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.154.129
--webserver_port=39127
--tserver_master_addrs=127.28.154.190:40041
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.154.129
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:11:04.395123 20904 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:11:04.396611 20904 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:11:04.422262 20966 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:11:04.423221 20970 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:11:04.428807 20904 server_base.cc:1034] running on GCE node
W20260811 09:11:04.430217 20974 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:11:04.431035 20904 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:11:04.435325 20904 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:11:04.438743 20904 hybrid_clock.cc:648] HybridClock initialized: now 1786439464438112 us; error 615 us; skew 500 ppm
I20260811 09:11:04.439249 20904 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:11:04 dist-test-slave-xt4k krb5kdc[19035](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439464, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:11:04.442937 20904 init.cc:377] Logged in from keytab as kudu/127.28.154.129@KRBTEST.COM (short username kudu)
I20260811 09:11:04.444895 20904 webserver.cc:460] Webserver started at http://127.28.154.129:39127/ using document root <none> and password file <none>
I20260811 09:11:04.445662 20904 fs_manager.cc:359] Metadata directory not provided
I20260811 09:11:04.445873 20904 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 09:11:04 dist-test-slave-xt4k krb5kdc[19035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439464, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.129@KRBTEST.COM for HTTP/127.28.154.148@KRBTEST.COM
I20260811 09:11:04.485100 20904 fs_manager.cc:709] Time spent opening directory manager: real 0.008s user 0.003s sys 0.001s
I20260811 09:11:04.488720 20986 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:11:04.490340 20904 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260811 09:11:04.490604 20904 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ts-0/wal
uuid: "028deeec53e24908a86608129742f470"
format_stamp: "Formatted at 2026-08-11 09:11:00 on dist-test-slave-xt4k"
server_key: "e42a6c820f84f525a1339ccf94c38a0a"
server_key_iv: "be15b752314cc2485bbc938d01380063"
server_key_version: "kuduclusterkey@0"
I20260811 09:11:04.492149 20904 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:11:04.523684 20904 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:11:04.529532 20904 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:11:04.529930 20904 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:11:04.534865 20904 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:11:04.541170 20904 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:11:04.541282 20904 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:11:04.541399 20904 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:11:04.541476 20904 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:11:04.643692 20904 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.129:33165
I20260811 09:11:04.643723 21184 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.129:33165 every 8 connection(s)
I20260811 09:11:04.645803 20904 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ts-0/data/info.pb
I20260811 09:11:04.649798 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 20904
I20260811 09:11:04.650638 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.28.154.148:50383/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.154.130:40537
--local_ip_for_outbound_sockets=127.28.154.130
--tserver_master_addrs=127.28.154.190:40041
--webserver_port=35631
--webserver_interface=127.28.154.130
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.130.keytab
--principal=kudu/127.28.154.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:35237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.28.154.148:50383/kms with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 09:11:04 dist-test-slave-xt4k krb5kdc[19035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439464, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.129@KRBTEST.COM for kudu/127.28.154.190@KRBTEST.COM
I20260811 09:11:04.680091 21190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:04.649706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.129:60509 (local address 127.28.154.190:40041)
0811 09:11:04.657215 (+ 7509us) server_negotiation.cc:206] Beginning negotiation
0811 09:11:04.657237 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0811 09:11:04.657288 (+ 51us) server_negotiation.cc:407] Connection header received
0811 09:11:04.657705 (+ 417us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:11:04.657720 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:11:04.658137 (+ 417us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:11:04.660498 (+ 2361us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:11:04.660983 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:04.662698 (+ 1715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:04.664526 (+ 1828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:04.665597 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:04.665811 (+ 214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:04.670250 (+ 4439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:11:04.670260 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:11:04.670281 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:11:04.670330 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:11:04.674091 (+ 3761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:11:04.675192 (+ 1101us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:11:04.675203 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:11:04.675211 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:11:04.675433 (+ 222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:11:04.676121 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:11:04.676126 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:11:04.676130 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:11:04.676939 (+ 809us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:11:04.677170 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0811 09:11:04.678211 (+ 1041us) server_negotiation.cc:299] Negotiation successful
0811 09:11:04.678641 (+ 430us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7262,"thread_start_us":7050,"threads_started":1}
I20260811 09:11:04.680154 21192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:04.649915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:40041 (local address 127.28.154.129:60509)
0811 09:11:04.651106 (+ 1191us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:04.651161 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 09:11:04.653771 (+ 2610us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:11:04.658777 (+ 5006us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:11:04.658790 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:11:04.659460 (+ 670us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:11:04.660135 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:11:04.660164 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:11:04.662886 (+ 2722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:11:04.662890 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:11:04.664282 (+ 1392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:11:04.664301 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:11:04.665890 (+ 1589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:11:04.665898 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:11:04.666173 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:04.667002 (+ 829us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:11:04.667038 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:11:04.669856 (+ 2818us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:11:04.674304 (+ 4448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:11:04.674316 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:11:04.674334 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:11:04.674954 (+ 620us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:11:04.675592 (+ 638us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:11:04.675671 (+ 79us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:11:04.675675 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:11:04.675968 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:11:04.677119 (+ 1151us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:11:04.677128 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:11:04.677579 (+ 451us) client_negotiation.cc:769] Sending connection context
0811 09:11:04.677913 (+ 334us) client_negotiation.cc:240] Negotiation successful
0811 09:11:04.678531 (+ 618us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":603,"thread_start_us":385,"threads_started":1}
I20260811 09:11:04.684603 21185 heartbeater.cc:347] Connected to a master server at 127.28.154.190:40041
I20260811 09:11:04.684991 21185 heartbeater.cc:464] Registering TS with master...
I20260811 09:11:04.685922 21185 heartbeater.cc:511] Master 127.28.154.190:40041 requested a full tablet report, sending...
W20260811 09:11:04.689550 21185 heartbeater.cc:644] Failed to heartbeat to 127.28.154.190:40041 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 09:11:04.689827 21185 heartbeater.cc:464] Registering TS with master...
I20260811 09:11:04.690161 21185 heartbeater.cc:511] Master 127.28.154.190:40041 requested a full tablet report, sending...
I20260811 09:11:04.691754 21185 heartbeater.cc:464] Registering TS with master...
I20260811 09:11:04.692070 21185 heartbeater.cc:511] Master 127.28.154.190:40041 requested a full tablet report, sending...
W20260811 09:11:04.693318 21185 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 09:11:05.067476 21193 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:11:05.067796 21193 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:11:05.067961 21193 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:11:05.075953 21193 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:11:05.076095 21193 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:11:05.076280 21193 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.154.130
I20260811 09:11:05.088058 21193 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:35237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.28.154.148:50383/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.130.keytab
--principal=kudu/127.28.154.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.154.130:40537
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.154.130
--webserver_port=35631
--tserver_master_addrs=127.28.154.190:40041
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.154.130
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:11:05.089643 21193 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:11:05.091243 21193 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:11:05.107254 21216 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:11:05.107292 21218 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:11:05.107673 21214 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:11:05.108199 21193 server_base.cc:1034] running on GCE node
I20260811 09:11:05.108929 21193 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:11:05.109966 21193 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:11:05.111258 21193 hybrid_clock.cc:648] HybridClock initialized: now 1786439465111194 us; error 41 us; skew 500 ppm
I20260811 09:11:05.111613 21193 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:11:05 dist-test-slave-xt4k krb5kdc[19035](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439465, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:11:05.115985 21193 init.cc:377] Logged in from keytab as kudu/127.28.154.130@KRBTEST.COM (short username kudu)
I20260811 09:11:05.148803 21193 webserver.cc:460] Webserver started at http://127.28.154.130:35631/ using document root <none> and password file <none>
I20260811 09:11:05.150029 21193 fs_manager.cc:359] Metadata directory not provided
I20260811 09:11:05.150291 21193 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 09:11:05 dist-test-slave-xt4k krb5kdc[19035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439465, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.130@KRBTEST.COM for HTTP/127.28.154.148@KRBTEST.COM
I20260811 09:11:05.196499 21193 fs_manager.cc:709] Time spent opening directory manager: real 0.019s user 0.009s sys 0.002s
I20260811 09:11:05.200404 21237 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:11:05.201894 21193 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 09:11:05.202069 21193 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ts-1/wal
uuid: "99dfb45b51a54908bc7312a5675b6899"
format_stamp: "Formatted at 2026-08-11 09:11:02 on dist-test-slave-xt4k"
server_key: "3e07135aa18402453dbfcdc57f865f60"
server_key_iv: "555121fc3c3a45fb01a5dbd45f423891"
server_key_version: "kuduclusterkey@0"
I20260811 09:11:05.203137 21193 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:11:05.224822 21193 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:11:05.229432 21193 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:11:05.229717 21193 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:11:05.230780 21193 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:11:05.232318 21193 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:11:05.232468 21193 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260811 09:11:05.232586 21193 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:11:05.232658 21193 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 09:11:05 dist-test-slave-xt4k krb5kdc[19035](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786439465, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:11:05.327026 21193 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.130:40537
I20260811 09:11:05.327075 21449 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.130:40537 every 8 connection(s)
I20260811 09:11:05.328619 21193 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ts-1/data/info.pb
I20260811 09:11:05.331885 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 21193
I20260811 09:11:05.332666 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.28.154.148:50383/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.154.131:45265
--local_ip_for_outbound_sockets=127.28.154.131
--tserver_master_addrs=127.28.154.190:40041
--webserver_port=43717
--webserver_interface=127.28.154.131
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.131.keytab
--principal=kudu/127.28.154.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:35237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.28.154.148:50383/kms with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 09:11:05 dist-test-slave-xt4k krb5kdc[19035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439465, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.130@KRBTEST.COM for kudu/127.28.154.190@KRBTEST.COM
I20260811 09:11:05.361265 21457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:05.336562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.130:44107 (local address 127.28.154.190:40041)
0811 09:11:05.341468 (+ 4906us) server_negotiation.cc:206] Beginning negotiation
0811 09:11:05.341473 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:11:05.341499 (+ 26us) server_negotiation.cc:407] Connection header received
0811 09:11:05.341557 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:11:05.341561 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:11:05.341628 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:11:05.341728 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:11:05.343965 (+ 2237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:05.345326 (+ 1361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:05.348014 (+ 2688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:05.348708 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:05.348845 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:05.353422 (+ 4577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:11:05.353428 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:11:05.353433 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:11:05.353473 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:11:05.356899 (+ 3426us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:11:05.357850 (+ 951us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:11:05.357855 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:11:05.357858 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:11:05.358008 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:11:05.358539 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:11:05.358543 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:11:05.358546 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:11:05.358893 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:11:05.359015 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 09:11:05.360688 (+ 1673us) server_negotiation.cc:299] Negotiation successful
0811 09:11:05.360902 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4776,"thread_start_us":4666,"threads_started":1}
I20260811 09:11:05.363124 21453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:05.333026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:40041 (local address 127.28.154.130:44107)
0811 09:11:05.338092 (+ 5066us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:05.338150 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 09:11:05.340659 (+ 2509us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:11:05.342142 (+ 1483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:11:05.342156 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:11:05.342838 (+ 682us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:11:05.343716 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:11:05.343746 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:11:05.345523 (+ 1777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:11:05.345529 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:11:05.347735 (+ 2206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:11:05.347755 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:11:05.349224 (+ 1469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:11:05.349232 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:11:05.349527 (+ 295us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:05.350338 (+ 811us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:11:05.350371 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:11:05.353139 (+ 2768us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:11:05.357067 (+ 3928us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:11:05.357077 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:11:05.357096 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:11:05.357695 (+ 599us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:11:05.358133 (+ 438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:11:05.358138 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:11:05.358141 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:11:05.358398 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:11:05.359102 (+ 704us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:11:05.359111 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:11:05.359624 (+ 513us) client_negotiation.cc:769] Sending connection context
0811 09:11:05.359935 (+ 311us) client_negotiation.cc:240] Negotiation successful
0811 09:11:05.361679 (+ 1744us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4747,"thread_start_us":4506,"threads_started":1}
I20260811 09:11:05.368709 21451 heartbeater.cc:347] Connected to a master server at 127.28.154.190:40041
I20260811 09:11:05.369062 21451 heartbeater.cc:464] Registering TS with master...
I20260811 09:11:05.370062 21451 heartbeater.cc:511] Master 127.28.154.190:40041 requested a full tablet report, sending...
I20260811 09:11:05.372462 20836 ts_manager.cc:194] Registered new tserver with Master: 99dfb45b51a54908bc7312a5675b6899 (127.28.154.130:40537)
I20260811 09:11:05.374732 20836 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.154.130@KRBTEST.COM'} at 127.28.154.130:44107
I20260811 09:11:05.693509 21185 heartbeater.cc:464] Registering TS with master...
I20260811 09:11:05.694047 21185 heartbeater.cc:511] Master 127.28.154.190:40041 requested a full tablet report, sending...
I20260811 09:11:05.695518 20836 ts_manager.cc:194] Registered new tserver with Master: 028deeec53e24908a86608129742f470 (127.28.154.129:33165)
I20260811 09:11:05.696884 20836 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.154.129@KRBTEST.COM'} at 127.28.154.129:60509
Aug 11 09:11:05 dist-test-slave-xt4k krb5kdc[19035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439465, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.190@KRBTEST.COM for HTTP/127.28.154.148@KRBTEST.COM
W20260811 09:11:05.818679 21459 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:11:05.819182 21459 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:11:05.819407 21459 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:11:05.831153 21459 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:11:05.831398 21459 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:11:05.831634 21459 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.154.131
I20260811 09:11:05.846488 21459 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:35237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.28.154.148:50383/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.131.keytab
--principal=kudu/127.28.154.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.154.131:45265
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.154.131
--webserver_port=43717
--tserver_master_addrs=127.28.154.190:40041
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.154.131
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:11:05.848376 21459 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:11:05.850144 21459 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:11:05.867574 21470 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:11:05.867630 21466 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:11:05.869911 21474 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:11:05.870482 21459 server_base.cc:1034] running on GCE node
I20260811 09:11:05.871320 21459 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:11:05.872756 21459 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:11:05.874024 21459 hybrid_clock.cc:648] HybridClock initialized: now 1786439465873920 us; error 80 us; skew 500 ppm
I20260811 09:11:05.874441 21459 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:11:05 dist-test-slave-xt4k krb5kdc[19035](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439465, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:11:05.881265 21459 init.cc:377] Logged in from keytab as kudu/127.28.154.131@KRBTEST.COM (short username kudu)
I20260811 09:11:05.885794 21459 webserver.cc:460] Webserver started at http://127.28.154.131:43717/ using document root <none> and password file <none>
I20260811 09:11:05.886704 21459 fs_manager.cc:359] Metadata directory not provided
I20260811 09:11:05.886857 21459 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 09:11:05 dist-test-slave-xt4k krb5kdc[19035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439465, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.131@KRBTEST.COM for HTTP/127.28.154.148@KRBTEST.COM
I20260811 09:11:05.911231 21459 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 09:11:05.926074 21494 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:11:05.927811 21459 fs_manager.cc:725] Time spent opening block manager: real 0.015s user 0.010s sys 0.004s
I20260811 09:11:05.928030 21459 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ts-2/wal
uuid: "2805ee604f62472f8e512ef3e1153cda"
format_stamp: "Formatted at 2026-08-11 09:11:02 on dist-test-slave-xt4k"
server_key: "025f4c88ae6effe63c329d5aa2d6fecb"
server_key_iv: "74e7664078104c09a9ac689393d81b70"
server_key_version: "kuduclusterkey@0"
I20260811 09:11:05.929167 21459 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:11:05.949227 21459 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:11:05.953739 21459 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:11:05.954046 21459 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:11:05.955166 21459 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:11:05.956763 21459 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:11:05.956864 21459 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:11:05.956988 21459 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:11:05.957057 21459 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:11:06.034502 21459 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.131:45265
I20260811 09:11:06.035034 21698 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.131:45265 every 8 connection(s)
I20260811 09:11:06.036685 21459 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ts-2/data/info.pb
I20260811 09:11:06.039422 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 21459
I20260811 09:11:06.082759 21705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:06.040621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37076 (local address 127.28.154.190:40041)
0811 09:11:06.060874 (+ 20253us) server_negotiation.cc:206] Beginning negotiation
0811 09:11:06.060881 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:11:06.060908 (+ 27us) server_negotiation.cc:407] Connection header received
0811 09:11:06.061003 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:11:06.061008 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:11:06.061125 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:11:06.061268 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:11:06.063796 (+ 2528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:06.065400 (+ 1604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:06.067847 (+ 2447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:06.069090 (+ 1243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:06.069229 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:06.072978 (+ 3749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:11:06.072985 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:11:06.072989 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:11:06.073037 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:11:06.077622 (+ 4585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:11:06.078751 (+ 1129us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:11:06.078758 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:11:06.078763 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:11:06.078955 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:11:06.079616 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:11:06.079622 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:11:06.079626 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:11:06.080088 (+ 462us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:11:06.081676 (+ 1588us) server_negotiation.cc:1035] Waiting for connection context
0811 09:11:06.081944 (+ 268us) server_negotiation.cc:299] Negotiation successful
0811 09:11:06.082283 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20107,"thread_start_us":19962,"threads_started":1}
Aug 11 09:11:06 dist-test-slave-xt4k krb5kdc[19035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439465, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.131@KRBTEST.COM for kudu/127.28.154.190@KRBTEST.COM
Aug 11 09:11:06 dist-test-slave-xt4k krb5kdc[19035](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439465, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.190@KRBTEST.COM for HTTP/127.28.154.148@KRBTEST.COM
I20260811 09:11:06.124291 21715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:06.064728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:40041 (local address 127.28.154.131:55239)
0811 09:11:06.097324 (+ 32596us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:06.097404 (+ 80us) client_negotiation.cc:174] Beginning negotiation
0811 09:11:06.100358 (+ 2954us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:11:06.101686 (+ 1328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:11:06.101700 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:11:06.102394 (+ 694us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:11:06.103311 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:11:06.103339 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:11:06.104480 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:11:06.104494 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:11:06.106545 (+ 2051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:11:06.106568 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:11:06.107816 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:11:06.107824 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:11:06.108237 (+ 413us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:06.109056 (+ 819us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:11:06.109088 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:11:06.114546 (+ 5458us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:11:06.118667 (+ 4121us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:11:06.118681 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:11:06.118708 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:11:06.119405 (+ 697us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:11:06.119995 (+ 590us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:11:06.120000 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:11:06.120004 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:11:06.120376 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:11:06.121194 (+ 818us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:11:06.121204 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:11:06.121715 (+ 511us) client_negotiation.cc:769] Sending connection context
0811 09:11:06.122056 (+ 341us) client_negotiation.cc:240] Negotiation successful
0811 09:11:06.122769 (+ 713us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32203,"thread_start_us":31939,"threads_started":1}
I20260811 09:11:06.125599 21705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:06.066518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.131:55239 (local address 127.28.154.190:40041)
0811 09:11:06.083317 (+ 16799us) server_negotiation.cc:206] Beginning negotiation
0811 09:11:06.083323 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:11:06.098153 (+ 14830us) server_negotiation.cc:407] Connection header received
0811 09:11:06.100732 (+ 2579us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:11:06.100739 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:11:06.100845 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:11:06.100957 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:11:06.103504 (+ 2547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:06.104226 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:06.106781 (+ 2555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:06.107464 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:06.108729 (+ 1265us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:06.114875 (+ 6146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:11:06.114881 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:11:06.114886 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:11:06.114933 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:11:06.118444 (+ 3511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:11:06.119570 (+ 1126us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:11:06.119575 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:11:06.119579 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:11:06.119826 (+ 247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:11:06.120544 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:11:06.120549 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:11:06.120553 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:11:06.120946 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:11:06.121098 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0811 09:11:06.124947 (+ 3849us) server_negotiation.cc:299] Negotiation successful
0811 09:11:06.125240 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16678,"thread_start_us":32624,"threads_started":1}
I20260811 09:11:06.127883 21699 heartbeater.cc:347] Connected to a master server at 127.28.154.190:40041
I20260811 09:11:06.128283 21699 heartbeater.cc:464] Registering TS with master...
I20260811 09:11:06.130185 21699 heartbeater.cc:511] Master 127.28.154.190:40041 requested a full tablet report, sending...
I20260811 09:11:06.133633 20836 ts_manager.cc:194] Registered new tserver with Master: 2805ee604f62472f8e512ef3e1153cda (127.28.154.131:45265)
I20260811 09:11:06.135676 20836 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.154.131@KRBTEST.COM'} at 127.28.154.131:55239
I20260811 09:11:06.145395 29290 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 09:11:06.198426 21705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:06.173166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37080 (local address 127.28.154.190:40041)
0811 09:11:06.173393 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0811 09:11:06.173401 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:11:06.179979 (+ 6578us) server_negotiation.cc:407] Connection header received
0811 09:11:06.180686 (+ 707us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:11:06.180694 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:11:06.180798 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:11:06.180883 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:11:06.183037 (+ 2154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:06.183821 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:06.185632 (+ 1811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:06.186345 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:06.186461 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:06.190612 (+ 4151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:11:06.190620 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:11:06.190626 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:11:06.190679 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:11:06.194193 (+ 3514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:11:06.195286 (+ 1093us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:11:06.195293 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:11:06.195297 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:11:06.195556 (+ 259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:11:06.196094 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:11:06.196100 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:11:06.196104 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:11:06.196591 (+ 487us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:11:06.196900 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0811 09:11:06.197772 (+ 872us) server_negotiation.cc:299] Negotiation successful
0811 09:11:06.198044 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 09:11:06.206252 20836 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37080:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 09:11:06.216960 20836 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 09:11:06.280912 21744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:06.244615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.130:40537 (local address 127.0.0.1:35002)
0811 09:11:06.262933 (+ 18318us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:06.263008 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0811 09:11:06.263245 (+ 237us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:11:06.266621 (+ 3376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:11:06.266630 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:11:06.266671 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:11:06.267129 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:11:06.267145 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:11:06.269244 (+ 2099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:11:06.269250 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:11:06.271699 (+ 2449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:11:06.271721 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:11:06.279674 (+ 7953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:11:06.279681 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:11:06.279936 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:06.280030 (+ 94us) client_negotiation.cc:769] Sending connection context
0811 09:11:06.280186 (+ 156us) client_negotiation.cc:240] Negotiation successful
0811 09:11:06.280435 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17968,"thread_start_us":19869,"threads_started":1}
I20260811 09:11:06.281888 21743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:06.244772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35002 (local address 127.28.154.130:40537)
0811 09:11:06.265481 (+ 20709us) server_negotiation.cc:206] Beginning negotiation
0811 09:11:06.265490 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 09:11:06.265522 (+ 32us) server_negotiation.cc:407] Connection header received
0811 09:11:06.265637 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:11:06.265645 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:11:06.265910 (+ 265us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:11:06.266141 (+ 231us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:11:06.267341 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:06.269064 (+ 1723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:06.271920 (+ 2856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:06.273515 (+ 1595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:06.273691 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:06.273888 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0811 09:11:06.281280 (+ 7392us) server_negotiation.cc:299] Negotiation successful
0811 09:11:06.281561 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20533,"thread_start_us":21560,"threads_started":1}
I20260811 09:11:06.287119 21332 tablet_service.cc:1467] Processing CreateTablet for tablet 2af8d09b833e449ca383dfcd1860193b (DEFAULT_TABLE table=test-table [id=7896b06f07e94c56985cedc2cbef0fe5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:11:06.288800 21740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:06.247561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.131:45265 (local address 127.0.0.1:43416)
0811 09:11:06.264825 (+ 17264us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:06.264846 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 09:11:06.264977 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:11:06.280415 (+ 15438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:11:06.280420 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:11:06.280451 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:11:06.280865 (+ 414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:11:06.280878 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:11:06.283763 (+ 2885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:11:06.283769 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:11:06.285824 (+ 2055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:11:06.285846 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:11:06.287828 (+ 1982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:11:06.287836 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:11:06.288124 (+ 288us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:06.288164 (+ 40us) client_negotiation.cc:769] Sending connection context
0811 09:11:06.288241 (+ 77us) client_negotiation.cc:240] Negotiation successful
0811 09:11:06.288434 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17095,"thread_start_us":14849,"threads_started":1}
I20260811 09:11:06.288899 21332 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2af8d09b833e449ca383dfcd1860193b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:11:06.290331 21745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:06.247834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43416 (local address 127.28.154.131:45265)
0811 09:11:06.279511 (+ 31677us) server_negotiation.cc:206] Beginning negotiation
0811 09:11:06.279521 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 09:11:06.279575 (+ 54us) server_negotiation.cc:407] Connection header received
0811 09:11:06.279692 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:11:06.279701 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:11:06.279974 (+ 273us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:11:06.280244 (+ 270us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:11:06.281836 (+ 1592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:06.283587 (+ 1751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:06.286082 (+ 2495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:06.287563 (+ 1481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:06.289531 (+ 1968us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:06.289738 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0811 09:11:06.289823 (+ 85us) server_negotiation.cc:299] Negotiation successful
0811 09:11:06.289994 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31480,"threads_started":1}
I20260811 09:11:06.300905 21589 tablet_service.cc:1467] Processing CreateTablet for tablet 2af8d09b833e449ca383dfcd1860193b (DEFAULT_TABLE table=test-table [id=7896b06f07e94c56985cedc2cbef0fe5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:11:06.302591 21589 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2af8d09b833e449ca383dfcd1860193b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:11:06.323843 21746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:06.265154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41860 (local address 127.28.154.129:33165)
0811 09:11:06.277701 (+ 12547us) server_negotiation.cc:206] Beginning negotiation
0811 09:11:06.277714 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 09:11:06.277756 (+ 42us) server_negotiation.cc:407] Connection header received
0811 09:11:06.277895 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:11:06.277906 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:11:06.278238 (+ 332us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:11:06.278503 (+ 265us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:11:06.279350 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:06.282933 (+ 3583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:06.320633 (+ 37700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:06.322206 (+ 1573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:06.322414 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:06.322631 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0811 09:11:06.323306 (+ 675us) server_negotiation.cc:299] Negotiation successful
0811 09:11:06.323513 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12304,"thread_start_us":13636,"threads_started":1}
I20260811 09:11:06.324738 21747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:06.271197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.129:33165 (local address 127.0.0.1:41860)
0811 09:11:06.273976 (+ 2779us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:06.273999 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 09:11:06.274183 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:11:06.278487 (+ 4304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:11:06.278492 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:11:06.278525 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:11:06.279036 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:11:06.279090 (+ 54us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:11:06.296471 (+ 17381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:11:06.296477 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:11:06.308353 (+ 11876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:11:06.308383 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:11:06.322867 (+ 14484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:11:06.322874 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:11:06.323074 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:06.323109 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 09:11:06.324157 (+ 1048us) client_negotiation.cc:240] Negotiation successful
0811 09:11:06.324327 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2621,"thread_start_us":2329,"threads_started":1}
I20260811 09:11:06.325358 21748 tablet_bootstrap.cc:492] T 2af8d09b833e449ca383dfcd1860193b P 99dfb45b51a54908bc7312a5675b6899: Bootstrap starting.
I20260811 09:11:06.329254 21748 tablet_bootstrap.cc:654] T 2af8d09b833e449ca383dfcd1860193b P 99dfb45b51a54908bc7312a5675b6899: Neither blocks nor log segments found. Creating new log.
I20260811 09:11:06.329169 21074 tablet_service.cc:1467] Processing CreateTablet for tablet 2af8d09b833e449ca383dfcd1860193b (DEFAULT_TABLE table=test-table [id=7896b06f07e94c56985cedc2cbef0fe5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:11:06.330541 21748 log.cc:824] T 2af8d09b833e449ca383dfcd1860193b P 99dfb45b51a54908bc7312a5675b6899: Log is configured to *not* fsync() on all Append() calls
I20260811 09:11:06.330919 21074 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2af8d09b833e449ca383dfcd1860193b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:11:06.341601 21750 tablet_bootstrap.cc:492] T 2af8d09b833e449ca383dfcd1860193b P 2805ee604f62472f8e512ef3e1153cda: Bootstrap starting.
I20260811 09:11:06.345661 21750 tablet_bootstrap.cc:654] T 2af8d09b833e449ca383dfcd1860193b P 2805ee604f62472f8e512ef3e1153cda: Neither blocks nor log segments found. Creating new log.
I20260811 09:11:06.346966 21750 log.cc:824] T 2af8d09b833e449ca383dfcd1860193b P 2805ee604f62472f8e512ef3e1153cda: Log is configured to *not* fsync() on all Append() calls
I20260811 09:11:06.350064 21748 tablet_bootstrap.cc:492] T 2af8d09b833e449ca383dfcd1860193b P 99dfb45b51a54908bc7312a5675b6899: No bootstrap required, opened a new log
I20260811 09:11:06.350419 21748 ts_tablet_manager.cc:1397] T 2af8d09b833e449ca383dfcd1860193b P 99dfb45b51a54908bc7312a5675b6899: Time spent bootstrapping tablet: real 0.025s user 0.009s sys 0.000s
I20260811 09:11:06.355227 21755 tablet_bootstrap.cc:492] T 2af8d09b833e449ca383dfcd1860193b P 028deeec53e24908a86608129742f470: Bootstrap starting.
I20260811 09:11:06.359128 21755 tablet_bootstrap.cc:654] T 2af8d09b833e449ca383dfcd1860193b P 028deeec53e24908a86608129742f470: Neither blocks nor log segments found. Creating new log.
I20260811 09:11:06.360519 21755 log.cc:824] T 2af8d09b833e449ca383dfcd1860193b P 028deeec53e24908a86608129742f470: Log is configured to *not* fsync() on all Append() calls
I20260811 09:11:06.363252 21750 tablet_bootstrap.cc:492] T 2af8d09b833e449ca383dfcd1860193b P 2805ee604f62472f8e512ef3e1153cda: No bootstrap required, opened a new log
I20260811 09:11:06.363584 21750 ts_tablet_manager.cc:1397] T 2af8d09b833e449ca383dfcd1860193b P 2805ee604f62472f8e512ef3e1153cda: Time spent bootstrapping tablet: real 0.022s user 0.004s sys 0.009s
I20260811 09:11:06.366097 21748 raft_consensus.cc:348] T 2af8d09b833e449ca383dfcd1860193b P 99dfb45b51a54908bc7312a5675b6899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99dfb45b51a54908bc7312a5675b6899" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 40537 } } peers { permanent_uuid: "028deeec53e24908a86608129742f470" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 33165 } } peers { permanent_uuid: "2805ee604f62472f8e512ef3e1153cda" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 45265 } }
I20260811 09:11:06.366736 21748 raft_consensus.cc:374] T 2af8d09b833e449ca383dfcd1860193b P 99dfb45b51a54908bc7312a5675b6899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:11:06.366894 21748 raft_consensus.cc:729] T 2af8d09b833e449ca383dfcd1860193b P 99dfb45b51a54908bc7312a5675b6899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99dfb45b51a54908bc7312a5675b6899, State: Initialized, Role: FOLLOWER
I20260811 09:11:06.367741 21748 consensus_queue.cc:260] T 2af8d09b833e449ca383dfcd1860193b P 99dfb45b51a54908bc7312a5675b6899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99dfb45b51a54908bc7312a5675b6899" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 40537 } } peers { permanent_uuid: "028deeec53e24908a86608129742f470" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 33165 } } peers { permanent_uuid: "2805ee604f62472f8e512ef3e1153cda" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 45265 } }
I20260811 09:11:06.374195 21750 raft_consensus.cc:348] T 2af8d09b833e449ca383dfcd1860193b P 2805ee604f62472f8e512ef3e1153cda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99dfb45b51a54908bc7312a5675b6899" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 40537 } } peers { permanent_uuid: "028deeec53e24908a86608129742f470" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 33165 } } peers { permanent_uuid: "2805ee604f62472f8e512ef3e1153cda" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 45265 } }
I20260811 09:11:06.374696 21750 raft_consensus.cc:374] T 2af8d09b833e449ca383dfcd1860193b P 2805ee604f62472f8e512ef3e1153cda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:11:06.374800 21750 raft_consensus.cc:729] T 2af8d09b833e449ca383dfcd1860193b P 2805ee604f62472f8e512ef3e1153cda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2805ee604f62472f8e512ef3e1153cda, State: Initialized, Role: FOLLOWER
I20260811 09:11:06.377198 21748 ts_tablet_manager.cc:1428] T 2af8d09b833e449ca383dfcd1860193b P 99dfb45b51a54908bc7312a5675b6899: Time spent starting tablet: real 0.027s user 0.016s sys 0.007s
I20260811 09:11:06.379001 21755 tablet_bootstrap.cc:492] T 2af8d09b833e449ca383dfcd1860193b P 028deeec53e24908a86608129742f470: No bootstrap required, opened a new log
I20260811 09:11:06.379421 21451 heartbeater.cc:503] Master 127.28.154.190:40041 was elected leader, sending a full tablet report...
I20260811 09:11:06.379606 21750 consensus_queue.cc:260] T 2af8d09b833e449ca383dfcd1860193b P 2805ee604f62472f8e512ef3e1153cda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99dfb45b51a54908bc7312a5675b6899" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 40537 } } peers { permanent_uuid: "028deeec53e24908a86608129742f470" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 33165 } } peers { permanent_uuid: "2805ee604f62472f8e512ef3e1153cda" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 45265 } }
I20260811 09:11:06.380074 21755 ts_tablet_manager.cc:1397] T 2af8d09b833e449ca383dfcd1860193b P 028deeec53e24908a86608129742f470: Time spent bootstrapping tablet: real 0.025s user 0.004s sys 0.004s
I20260811 09:11:06.387128 21699 heartbeater.cc:503] Master 127.28.154.190:40041 was elected leader, sending a full tablet report...
I20260811 09:11:06.387871 21750 ts_tablet_manager.cc:1428] T 2af8d09b833e449ca383dfcd1860193b P 2805ee604f62472f8e512ef3e1153cda: Time spent starting tablet: real 0.024s user 0.014s sys 0.007s
I20260811 09:11:06.391654 21755 raft_consensus.cc:348] T 2af8d09b833e449ca383dfcd1860193b P 028deeec53e24908a86608129742f470 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99dfb45b51a54908bc7312a5675b6899" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 40537 } } peers { permanent_uuid: "028deeec53e24908a86608129742f470" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 33165 } } peers { permanent_uuid: "2805ee604f62472f8e512ef3e1153cda" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 45265 } }
I20260811 09:11:06.392300 21755 raft_consensus.cc:374] T 2af8d09b833e449ca383dfcd1860193b P 028deeec53e24908a86608129742f470 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:11:06.392477 21755 raft_consensus.cc:729] T 2af8d09b833e449ca383dfcd1860193b P 028deeec53e24908a86608129742f470 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 028deeec53e24908a86608129742f470, State: Initialized, Role: FOLLOWER
I20260811 09:11:06.393252 21755 consensus_queue.cc:260] T 2af8d09b833e449ca383dfcd1860193b P 028deeec53e24908a86608129742f470 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99dfb45b51a54908bc7312a5675b6899" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 40537 } } peers { permanent_uuid: "028deeec53e24908a86608129742f470" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 33165 } } peers { permanent_uuid: "2805ee604f62472f8e512ef3e1153cda" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 45265 } }
I20260811 09:11:06.403992 21185 heartbeater.cc:503] Master 127.28.154.190:40041 was elected leader, sending a full tablet report...
I20260811 09:11:06.405139 21755 ts_tablet_manager.cc:1428] T 2af8d09b833e449ca383dfcd1860193b P 028deeec53e24908a86608129742f470: Time spent starting tablet: real 0.025s user 0.016s sys 0.007s
I20260811 09:11:06.405753 21764 raft_consensus.cc:482] T 2af8d09b833e449ca383dfcd1860193b P 028deeec53e24908a86608129742f470 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:11:06.406067 21764 raft_consensus.cc:504] T 2af8d09b833e449ca383dfcd1860193b P 028deeec53e24908a86608129742f470 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99dfb45b51a54908bc7312a5675b6899" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 40537 } } peers { permanent_uuid: "028deeec53e24908a86608129742f470" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 33165 } } peers { permanent_uuid: "2805ee604f62472f8e512ef3e1153cda" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 45265 } }
I20260811 09:11:06.408418 21764 leader_election.cc:288] T 2af8d09b833e449ca383dfcd1860193b P 028deeec53e24908a86608129742f470 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99dfb45b51a54908bc7312a5675b6899 (127.28.154.130:40537), 2805ee604f62472f8e512ef3e1153cda (127.28.154.131:45265)
I20260811 09:11:06.428608 21743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:06.411441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.129:34501 (local address 127.28.154.130:40537)
0811 09:11:06.411684 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0811 09:11:06.411690 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:11:06.419816 (+ 8126us) server_negotiation.cc:407] Connection header received
0811 09:11:06.420042 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:11:06.420048 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:11:06.420135 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:11:06.420237 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:11:06.423123 (+ 2886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:06.424208 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:06.426719 (+ 2511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:06.427562 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:06.427726 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:06.427803 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 09:11:06.428176 (+ 373us) server_negotiation.cc:299] Negotiation successful
0811 09:11:06.428320 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260811 09:11:06.428951 21776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:06.409293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.130:40537 (local address 127.28.154.129:34501)
0811 09:11:06.419699 (+ 10406us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:06.419716 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 09:11:06.419872 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:11:06.422487 (+ 2615us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:11:06.422492 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:11:06.422521 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:11:06.422891 (+ 370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:11:06.422903 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:11:06.424418 (+ 1515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:11:06.424423 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:11:06.426507 (+ 2084us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:11:06.426520 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:11:06.427793 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:11:06.427800 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:11:06.427981 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:06.428018 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 09:11:06.428115 (+ 97us) client_negotiation.cc:240] Negotiation successful
0811 09:11:06.428263 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10280,"thread_start_us":10248,"threads_started":1}
I20260811 09:11:06.429555 21745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:06.409589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.129:44939 (local address 127.28.154.131:45265)
0811 09:11:06.409808 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0811 09:11:06.409815 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:11:06.419503 (+ 9688us) server_negotiation.cc:407] Connection header received
0811 09:11:06.421341 (+ 1838us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:11:06.421346 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:11:06.421427 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:11:06.421509 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:11:06.423738 (+ 2229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:06.424949 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:06.427443 (+ 2494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:06.428508 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:06.428661 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:06.428729 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 09:11:06.429187 (+ 458us) server_negotiation.cc:299] Negotiation successful
0811 09:11:06.429316 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 09:11:06.429659 21777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:06.409048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.131:45265 (local address 127.28.154.129:44939)
0811 09:11:06.419328 (+ 10280us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:06.419355 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 09:11:06.421153 (+ 1798us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:11:06.422897 (+ 1744us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:11:06.422901 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:11:06.422929 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:11:06.423537 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:11:06.423550 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:11:06.425129 (+ 1579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:11:06.425134 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:11:06.427258 (+ 2124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:11:06.427276 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:11:06.428882 (+ 1606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:11:06.428888 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:11:06.429033 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:06.429056 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 09:11:06.429138 (+ 82us) client_negotiation.cc:240] Negotiation successful
0811 09:11:06.429263 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10097,"mutex_wait_us":6,"thread_start_us":10369,"threads_started":1}
I20260811 09:11:06.430593 21623 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2af8d09b833e449ca383dfcd1860193b" candidate_uuid: "028deeec53e24908a86608129742f470" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2805ee604f62472f8e512ef3e1153cda" is_pre_election: true
I20260811 09:11:06.430644 21369 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2af8d09b833e449ca383dfcd1860193b" candidate_uuid: "028deeec53e24908a86608129742f470" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99dfb45b51a54908bc7312a5675b6899" is_pre_election: true
I20260811 09:11:06.431111 21623 raft_consensus.cc:2454] T 2af8d09b833e449ca383dfcd1860193b P 2805ee604f62472f8e512ef3e1153cda [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 028deeec53e24908a86608129742f470 in term 0.
I20260811 09:11:06.431214 21369 raft_consensus.cc:2454] T 2af8d09b833e449ca383dfcd1860193b P 99dfb45b51a54908bc7312a5675b6899 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 028deeec53e24908a86608129742f470 in term 0.
I20260811 09:11:06.431793 20992 leader_election.cc:302] T 2af8d09b833e449ca383dfcd1860193b P 028deeec53e24908a86608129742f470 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 028deeec53e24908a86608129742f470, 2805ee604f62472f8e512ef3e1153cda; no voters:
I20260811 09:11:06.432214 21773 raft_consensus.cc:2781] T 2af8d09b833e449ca383dfcd1860193b P 028deeec53e24908a86608129742f470 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 09:11:06.432384 21773 raft_consensus.cc:482] T 2af8d09b833e449ca383dfcd1860193b P 028deeec53e24908a86608129742f470 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 09:11:06.432523 21773 raft_consensus.cc:3037] T 2af8d09b833e449ca383dfcd1860193b P 028deeec53e24908a86608129742f470 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:11:06.435122 21773 raft_consensus.cc:504] T 2af8d09b833e449ca383dfcd1860193b P 028deeec53e24908a86608129742f470 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99dfb45b51a54908bc7312a5675b6899" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 40537 } } peers { permanent_uuid: "028deeec53e24908a86608129742f470" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 33165 } } peers { permanent_uuid: "2805ee604f62472f8e512ef3e1153cda" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 45265 } }
I20260811 09:11:06.435715 21773 leader_election.cc:288] T 2af8d09b833e449ca383dfcd1860193b P 028deeec53e24908a86608129742f470 [CANDIDATE]: Term 1 election: Requested vote from peers 99dfb45b51a54908bc7312a5675b6899 (127.28.154.130:40537), 2805ee604f62472f8e512ef3e1153cda (127.28.154.131:45265)
I20260811 09:11:06.436313 21623 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2af8d09b833e449ca383dfcd1860193b" candidate_uuid: "028deeec53e24908a86608129742f470" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2805ee604f62472f8e512ef3e1153cda"
I20260811 09:11:06.436386 21369 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2af8d09b833e449ca383dfcd1860193b" candidate_uuid: "028deeec53e24908a86608129742f470" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99dfb45b51a54908bc7312a5675b6899"
I20260811 09:11:06.436573 21623 raft_consensus.cc:3037] T 2af8d09b833e449ca383dfcd1860193b P 2805ee604f62472f8e512ef3e1153cda [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:11:06.436582 21369 raft_consensus.cc:3037] T 2af8d09b833e449ca383dfcd1860193b P 99dfb45b51a54908bc7312a5675b6899 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:11:06.439730 21369 raft_consensus.cc:2454] T 2af8d09b833e449ca383dfcd1860193b P 99dfb45b51a54908bc7312a5675b6899 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 028deeec53e24908a86608129742f470 in term 1.
I20260811 09:11:06.439730 21623 raft_consensus.cc:2454] T 2af8d09b833e449ca383dfcd1860193b P 2805ee604f62472f8e512ef3e1153cda [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 028deeec53e24908a86608129742f470 in term 1.
I20260811 09:11:06.440161 20992 leader_election.cc:302] T 2af8d09b833e449ca383dfcd1860193b P 028deeec53e24908a86608129742f470 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 028deeec53e24908a86608129742f470, 2805ee604f62472f8e512ef3e1153cda; no voters:
I20260811 09:11:06.440452 21773 raft_consensus.cc:2781] T 2af8d09b833e449ca383dfcd1860193b P 028deeec53e24908a86608129742f470 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:11:06.440636 21773 raft_consensus.cc:686] T 2af8d09b833e449ca383dfcd1860193b P 028deeec53e24908a86608129742f470 [term 1 LEADER]: Becoming Leader. State: Replica: 028deeec53e24908a86608129742f470, State: Running, Role: LEADER
I20260811 09:11:06.441197 21773 consensus_queue.cc:237] T 2af8d09b833e449ca383dfcd1860193b P 028deeec53e24908a86608129742f470 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99dfb45b51a54908bc7312a5675b6899" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 40537 } } peers { permanent_uuid: "028deeec53e24908a86608129742f470" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 33165 } } peers { permanent_uuid: "2805ee604f62472f8e512ef3e1153cda" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 45265 } }
I20260811 09:11:06.446578 20836 catalog_manager.cc:5495] T 2af8d09b833e449ca383dfcd1860193b P 028deeec53e24908a86608129742f470 reported cstate change: term changed from 0 to 1, leader changed from <none> to 028deeec53e24908a86608129742f470 (127.28.154.129). New cstate: current_term: 1 leader_uuid: "028deeec53e24908a86608129742f470" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99dfb45b51a54908bc7312a5675b6899" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 40537 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "028deeec53e24908a86608129742f470" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 33165 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2805ee604f62472f8e512ef3e1153cda" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 45265 } health_report { overall_health: UNKNOWN } } }
I20260811 09:11:06.493710 21746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:06.487514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41876 (local address 127.28.154.129:33165)
0811 09:11:06.487697 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0811 09:11:06.487701 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:11:06.487718 (+ 17us) server_negotiation.cc:407] Connection header received
0811 09:11:06.487809 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:11:06.487813 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:11:06.487891 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:11:06.488018 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:11:06.488471 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:06.489239 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:06.491211 (+ 1972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:06.491635 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:06.491727 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:06.492385 (+ 658us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:11:06.492965 (+ 580us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:11:06.493091 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0811 09:11:06.493317 (+ 226us) server_negotiation.cc:299] Negotiation successful
0811 09:11:06.493415 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 09:11:06.502987 21623 raft_consensus.cc:1261] T 2af8d09b833e449ca383dfcd1860193b P 2805ee604f62472f8e512ef3e1153cda [term 1 FOLLOWER]: Refusing update from remote peer 028deeec53e24908a86608129742f470: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:11:06.502985 21369 raft_consensus.cc:1261] T 2af8d09b833e449ca383dfcd1860193b P 99dfb45b51a54908bc7312a5675b6899 [term 1 FOLLOWER]: Refusing update from remote peer 028deeec53e24908a86608129742f470: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:11:06.504171 21773 consensus_queue.cc:1035] T 2af8d09b833e449ca383dfcd1860193b P 028deeec53e24908a86608129742f470 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99dfb45b51a54908bc7312a5675b6899" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 40537 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:11:06.504528 21764 consensus_queue.cc:1035] T 2af8d09b833e449ca383dfcd1860193b P 028deeec53e24908a86608129742f470 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2805ee604f62472f8e512ef3e1153cda" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 45265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:11:06.555729 20836 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37080:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:11:06.556059 20836 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37080:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:11:06.560679 20836 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 81cb27d79a0b47458e95c71620c20cc2: Sending DeleteTablet for 3 replicas of tablet 2af8d09b833e449ca383dfcd1860193b
I20260811 09:11:06.562119 21332 tablet_service.cc:1514] Processing DeleteTablet for tablet 2af8d09b833e449ca383dfcd1860193b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:11:06 UTC) from {username='kudu', principal='kudu/127.28.154.190@KRBTEST.COM'} at 127.0.0.1:35002
I20260811 09:11:06.562400 21589 tablet_service.cc:1514] Processing DeleteTablet for tablet 2af8d09b833e449ca383dfcd1860193b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:11:06 UTC) from {username='kudu', principal='kudu/127.28.154.190@KRBTEST.COM'} at 127.0.0.1:43416
I20260811 09:11:06.563131 21074 tablet_service.cc:1514] Processing DeleteTablet for tablet 2af8d09b833e449ca383dfcd1860193b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:11:06 UTC) from {username='kudu', principal='kudu/127.28.154.190@KRBTEST.COM'} at 127.0.0.1:41860
I20260811 09:11:06.574645 21808 tablet_replica.cc:330] T 2af8d09b833e449ca383dfcd1860193b P 99dfb45b51a54908bc7312a5675b6899: stopping tablet replica
I20260811 09:11:06.575309 21808 raft_consensus.cc:2229] T 2af8d09b833e449ca383dfcd1860193b P 99dfb45b51a54908bc7312a5675b6899 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:11:06.576189 21808 raft_consensus.cc:2258] T 2af8d09b833e449ca383dfcd1860193b P 99dfb45b51a54908bc7312a5675b6899 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:11:06.578437 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 20904
I20260811 09:11:06.580112 21808 ts_tablet_manager.cc:1905] T 2af8d09b833e449ca383dfcd1860193b P 99dfb45b51a54908bc7312a5675b6899: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:11:06.588095 21808 ts_tablet_manager.cc:1918] T 2af8d09b833e449ca383dfcd1860193b P 99dfb45b51a54908bc7312a5675b6899: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:11:06.588455 21808 log.cc:1196] T 2af8d09b833e449ca383dfcd1860193b P 99dfb45b51a54908bc7312a5675b6899: Deleting WAL directory at /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ts-1/wal/wals/2af8d09b833e449ca383dfcd1860193b
I20260811 09:11:06.589178 21808 ts_tablet_manager.cc:1939] T 2af8d09b833e449ca383dfcd1860193b P 99dfb45b51a54908bc7312a5675b6899: Deleting consensus metadata
I20260811 09:11:06.591646 20815 catalog_manager.cc:4841] TS 99dfb45b51a54908bc7312a5675b6899 (127.28.154.130:40537): tablet 2af8d09b833e449ca383dfcd1860193b (table test-table [id=7896b06f07e94c56985cedc2cbef0fe5]) successfully deleted
I20260811 09:11:06.604725 21746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:06.580154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41892 (local address 127.28.154.129:33165)
0811 09:11:06.580352 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0811 09:11:06.580358 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:11:06.580379 (+ 21us) server_negotiation.cc:407] Connection header received
0811 09:11:06.580448 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:11:06.580453 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:11:06.580597 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:11:06.580716 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:11:06.582871 (+ 2155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:06.584145 (+ 1274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:06.588008 (+ 3863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:06.588900 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:06.589067 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:06.595114 (+ 6047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:11:06.595141 (+ 27us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:11:06.595164 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:11:06.595251 (+ 87us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:11:06.599844 (+ 4593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:11:06.601008 (+ 1164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:11:06.601019 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:11:06.601022 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:11:06.601248 (+ 226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:11:06.601765 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:11:06.601771 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:11:06.601774 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:11:06.602309 (+ 535us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:11:06.602701 (+ 392us) server_negotiation.cc:1035] Waiting for connection context
0811 09:11:06.604027 (+ 1326us) server_negotiation.cc:299] Negotiation successful
0811 09:11:06.604273 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 09:11:06.605811 21179 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:11:06.606772 21809 tablet_replica.cc:330] T 2af8d09b833e449ca383dfcd1860193b P 2805ee604f62472f8e512ef3e1153cda: stopping tablet replica
I20260811 09:11:06.607323 21809 raft_consensus.cc:2229] T 2af8d09b833e449ca383dfcd1860193b P 2805ee604f62472f8e512ef3e1153cda [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:11:06.607971 21809 raft_consensus.cc:2258] T 2af8d09b833e449ca383dfcd1860193b P 2805ee604f62472f8e512ef3e1153cda [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:11:06.610857 21809 ts_tablet_manager.cc:1905] T 2af8d09b833e449ca383dfcd1860193b P 2805ee604f62472f8e512ef3e1153cda: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:11:06.619273 21809 ts_tablet_manager.cc:1918] T 2af8d09b833e449ca383dfcd1860193b P 2805ee604f62472f8e512ef3e1153cda: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:11:06.619519 21809 log.cc:1196] T 2af8d09b833e449ca383dfcd1860193b P 2805ee604f62472f8e512ef3e1153cda: Deleting WAL directory at /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786439250192320-29290-0/minicluster-data/ts-2/wal/wals/2af8d09b833e449ca383dfcd1860193b
I20260811 09:11:06.620165 21809 ts_tablet_manager.cc:1939] T 2af8d09b833e449ca383dfcd1860193b P 2805ee604f62472f8e512ef3e1153cda: Deleting consensus metadata
I20260811 09:11:06.622329 20812 catalog_manager.cc:4841] TS 2805ee604f62472f8e512ef3e1153cda (127.28.154.131:45265): tablet 2af8d09b833e449ca383dfcd1860193b (table test-table [id=7896b06f07e94c56985cedc2cbef0fe5]) successfully deleted
I20260811 09:11:06.720805 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 20904
W20260811 09:11:06.767134 20815 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.28.154.129:33165 (error 108)
I20260811 09:11:06.767428 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 21193
I20260811 09:11:06.768991 21747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:06.768521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.129:33165 (local address 127.0.0.1:41904)
0811 09:11:06.768729 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:06.768836 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.129:33165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
W20260811 09:11:06.769277 20815 catalog_manager.cc:4584] TS 028deeec53e24908a86608129742f470 (127.28.154.129:33165): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2af8d09b833e449ca383dfcd1860193b: Network error: Client connection negotiation failed: client connection to 127.28.154.129:33165: connect: Connection refused (error 111)
I20260811 09:11:06.782470 21743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:06.768220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35014 (local address 127.28.154.130:40537)
0811 09:11:06.768443 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0811 09:11:06.768449 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:11:06.768564 (+ 115us) server_negotiation.cc:407] Connection header received
0811 09:11:06.768787 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:11:06.768793 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:11:06.768879 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:11:06.769011 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:11:06.770500 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:06.771611 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:06.772993 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:06.773614 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:06.773751 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:06.776472 (+ 2721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:11:06.776489 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:11:06.776507 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:11:06.776544 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:11:06.779459 (+ 2915us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:11:06.780203 (+ 744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:11:06.780211 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:11:06.780213 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:11:06.780503 (+ 290us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:11:06.780897 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:11:06.780902 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:11:06.780906 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:11:06.781352 (+ 446us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:11:06.781644 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0811 09:11:06.781922 (+ 278us) server_negotiation.cc:299] Negotiation successful
0811 09:11:06.782100 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 09:11:06.783143 21439 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:11:06.823083 21747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:06.822590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.129:33165 (local address 127.0.0.1:41910)
0811 09:11:06.822812 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:06.822922 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.129:33165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260811 09:11:06.824102 21747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:06.823745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.129:33165 (local address 127.0.0.1:41918)
0811 09:11:06.823896 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:06.823967 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.129:33165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260811 09:11:06.875602 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 21193
I20260811 09:11:06.884850 21747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:06.884292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.129:33165 (local address 127.0.0.1:41922)
0811 09:11:06.884535 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:06.884628 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.129:33165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260811 09:11:06.885943 21747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:06.885603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.129:33165 (local address 127.0.0.1:41926)
0811 09:11:06.885749 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:06.885822 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.129:33165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260811 09:11:06.902902 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 21459
I20260811 09:11:06.916649 21745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:06.903804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43432 (local address 127.28.154.131:45265)
0811 09:11:06.904001 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0811 09:11:06.904006 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:11:06.904045 (+ 39us) server_negotiation.cc:407] Connection header received
0811 09:11:06.904192 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:11:06.904197 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:11:06.904271 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:11:06.904405 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:11:06.905752 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:06.906473 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:06.907701 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:06.908181 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:06.908295 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:06.910873 (+ 2578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:11:06.910889 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:11:06.910907 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:11:06.910941 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:11:06.913651 (+ 2710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:11:06.914301 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:11:06.914308 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:11:06.914311 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:11:06.914441 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:11:06.914813 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:11:06.914819 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:11:06.914822 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:11:06.915216 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:11:06.915531 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0811 09:11:06.915811 (+ 280us) server_negotiation.cc:299] Negotiation successful
0811 09:11:06.916041 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 09:11:06.917332 21692 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:11:06.966966 21747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:06.966465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.129:33165 (local address 127.0.0.1:41932)
0811 09:11:06.966688 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:06.966783 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.129:33165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260811 09:11:06.968011 21747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:06.967617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.129:33165 (local address 127.0.0.1:41946)
0811 09:11:06.967827 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:06.967901 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.129:33165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
I20260811 09:11:07.006843 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 21459
I20260811 09:11:07.028198 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 20777
I20260811 09:11:07.029564 20895 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:11:07.142969 21747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:07.142372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.129:33165 (local address 127.0.0.1:41950)
0811 09:11:07.142603 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:07.142762 (+ 159us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.129:33165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260811 09:11:07.143272 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 20777
I20260811 09:11:07.171868 29290 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260811 09:11:07.507728 29290 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260811 09:11:07.507931 29290 mini_ranger.cc:66] Stopping Ranger...
I20260811 09:11:07.853878 29290 mini_ranger.cc:68] Stopped Ranger
2026-08-11 09:11:07.854 UTC [19089] LOG: received smart shutdown request
2026-08-11 09:11:07.857 UTC [19089] LOG: background worker "logical replication launcher" (PID 19102) exited with exit code 1
2026-08-11 09:11:07.858 UTC [19095] LOG: shutting down
2026-08-11 09:11:08.059 UTC [19089] LOG: database system is shut down
2026-08-11T09:11:08Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (57448 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439250192320-29290-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:11:08 dist-test-slave-xt4k krb5kdc[21837](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:11:08 dist-test-slave-xt4k krb5kdc[21837](info): set up 2 sockets
Aug 11 09:11:08 dist-test-slave-xt4k krb5kdc[21837](info): commencing operation
krb5kdc: starting...
W20260811 09:11:09.470247 29290 mini_kdc.cc:121] Time spent starting KDC: real 1.252s user 0.000s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439250192320-29290-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439250192320-29290-0/krb5kdc/test-user.keytab.
Aug 11 09:11:09 dist-test-slave-xt4k krb5kdc[21837](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439469, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:11:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:11:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.154.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.154.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.154.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab.
Entry for principal kudu/127.28.154.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab.
I20260811 09:11:09.760509 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439250192320-29290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439250192320-29290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439250192320-29290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.154.190:44501
--webserver_interface=127.28.154.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab
--principal=kudu/127.28.154.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:34593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.154.190:44501,127.28.154.189:33105,127.28.154.188:38397
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:11:10.103750 21859 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:11:10.104130 21859 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:11:10.104274 21859 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:11:10.113060 21859 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:11:10.113226 21859 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:11:10.113319 21859 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:11:10.113478 21859 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:11:10.113607 21859 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:11:10.124670 21859 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:34593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439250192320-29290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439250192320-29290-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.154.190:44501,127.28.154.189:33105,127.28.154.188:38397
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab
--principal=kudu/127.28.154.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.154.190:44501
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.154.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439250192320-29290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:11:10.126173 21859 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:11:10.127565 21859 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:11:10.140676 21869 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:11:10.140676 21873 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:11:10.141451 21868 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:11:10.141950 21859 server_base.cc:1034] running on GCE node
I20260811 09:11:10.142786 21859 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:11:10.144263 21859 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:11:10.145536 21859 hybrid_clock.cc:648] HybridClock initialized: now 1786439470145488 us; error 32 us; skew 500 ppm
I20260811 09:11:10.145931 21859 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:11:10 dist-test-slave-xt4k krb5kdc[21837](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439470, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:11:10.149644 21859 init.cc:377] Logged in from keytab as kudu/127.28.154.190@KRBTEST.COM (short username kudu)
I20260811 09:11:10.151294 21859 webserver.cc:460] Webserver started at http://127.28.154.190:41327/ using document root <none> and password file <none>
I20260811 09:11:10.152017 21859 fs_manager.cc:359] Metadata directory not provided
I20260811 09:11:10.152139 21859 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:11:10.152511 21859 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:11:10.155565 21859 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439250192320-29290-0/minicluster-data/master-0/data/instance:
uuid: "e8d3967ffc3e4bda924776881f306353"
format_stamp: "Formatted at 2026-08-11 09:11:10 on dist-test-slave-xt4k"
server_key: "b2d7344f877f3c0b2aa9f1804c732d63"
server_key_iv: "d9d5e9286140daaaaf82e5e4eea9bd82"
server_key_version: "encryptionkey@0"
I20260811 09:11:10.156426 21859 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439250192320-29290-0/minicluster-data/master-0/wal/instance:
uuid: "e8d3967ffc3e4bda924776881f306353"
format_stamp: "Formatted at 2026-08-11 09:11:10 on dist-test-slave-xt4k"
server_key: "b2d7344f877f3c0b2aa9f1804c732d63"
server_key_iv: "d9d5e9286140daaaaf82e5e4eea9bd82"
server_key_version: "encryptionkey@0"
I20260811 09:11:10.162067 21859 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260811 09:11:10.165829 21883 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:11:10.167331 21859 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 09:11:10.167554 21859 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439250192320-29290-0/minicluster-data/master-0/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439250192320-29290-0/minicluster-data/master-0/wal
uuid: "e8d3967ffc3e4bda924776881f306353"
format_stamp: "Formatted at 2026-08-11 09:11:10 on dist-test-slave-xt4k"
server_key: "b2d7344f877f3c0b2aa9f1804c732d63"
server_key_iv: "d9d5e9286140daaaaf82e5e4eea9bd82"
server_key_version: "encryptionkey@0"
I20260811 09:11:10.167757 21859 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439250192320-29290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439250192320-29290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439250192320-29290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:11:10.193120 21859 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:11:10.197988 21859 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:11:10.198297 21859 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:11:10.217566 21859 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.190:44501
I20260811 09:11:10.217612 21976 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.190:44501 every 8 connection(s)
I20260811 09:11:10.218972 21859 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
I20260811 09:11:10.219625 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 21859
I20260811 09:11:10.219884 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439250192320-29290-0/minicluster-data/master-0/wal/instance
I20260811 09:11:10.220741 29290 external_mini_cluster.cc:1428] Setting key 98fd1e65ad5516210083dbaa66590749
I20260811 09:11:10.223225 21980 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:11:10.230232 21980 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.154.190" port: 44501 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 09:11:10 dist-test-slave-xt4k krb5kdc[21837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439470, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.190@KRBTEST.COM for kudu/127.28.154.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.28.154.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.154.189@KRBTEST.COM" created.
I20260811 09:11:10.255226 21986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:10.233785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:44501 (local address 127.0.0.1:43550)
0811 09:11:10.234711 (+ 926us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:10.234753 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 09:11:10.236998 (+ 2245us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:11:10.237874 (+ 876us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:11:10.237881 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:11:10.238617 (+ 736us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:11:10.239006 (+ 389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:11:10.239026 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:11:10.240638 (+ 1612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:11:10.240642 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:11:10.241754 (+ 1112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:11:10.241767 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:11:10.242865 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:11:10.242873 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:11:10.243070 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:10.243825 (+ 755us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:11:10.243850 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:11:10.246651 (+ 2801us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:11:10.250612 (+ 3961us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:11:10.250624 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:11:10.250636 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:11:10.251157 (+ 521us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:11:10.251685 (+ 528us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:11:10.251689 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:11:10.251691 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:11:10.251859 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:11:10.252890 (+ 1031us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:11:10.252899 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:11:10.253137 (+ 238us) client_negotiation.cc:769] Sending connection context
0811 09:11:10.253384 (+ 247us) client_negotiation.cc:240] Negotiation successful
0811 09:11:10.254133 (+ 749us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":652,"mutex_wait_us":266,"thread_start_us":418,"threads_started":1}
I20260811 09:11:10.255252 21990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:10.235261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43550 (local address 127.28.154.190:44501)
0811 09:11:10.236017 (+ 756us) server_negotiation.cc:206] Beginning negotiation
0811 09:11:10.236028 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 09:11:10.236099 (+ 71us) server_negotiation.cc:407] Connection header received
0811 09:11:10.237382 (+ 1283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:11:10.237388 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:11:10.237643 (+ 255us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:11:10.237836 (+ 193us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:11:10.239207 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:10.240490 (+ 1283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:10.241891 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:10.242485 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:10.242832 (+ 347us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:10.246932 (+ 4100us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:11:10.246944 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:11:10.246960 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:11:10.247000 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:11:10.250434 (+ 3434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:11:10.251318 (+ 884us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:11:10.251330 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:11:10.251334 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:11:10.251548 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:11:10.251974 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:11:10.251979 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:11:10.251981 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:11:10.252732 (+ 751us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:11:10.252947 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0811 09:11:10.253558 (+ 611us) server_negotiation.cc:299] Negotiation successful
0811 09:11:10.253957 (+ 399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":545,"thread_start_us":349,"threads_started":1}
I20260811 09:11:10.258330 21980 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.154.189" port: 33105 } has no permanent_uuid. Determining permanent_uuid...
I20260811 09:11:10.259598 21986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:10.259133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.189:33105 (local address 127.0.0.1:59968)
0811 09:11:10.259327 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:10.259458 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.189:33105: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260811 09:11:10.259930 21888 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.28.154.189:33105: connect: Connection refused (error 111)
I20260811 09:11:10.261639 21986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:10.261251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.189:33105 (local address 127.0.0.1:59982)
0811 09:11:10.261421 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:10.261502 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.189:33105: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260811 09:11:10.261837 21980 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.154.189:33105: Network error: Client connection negotiation failed: client connection to 127.28.154.189:33105: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.154.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.189.keytab.
Entry for principal kudu/127.28.154.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.189.keytab.
I20260811 09:11:10.284278 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439250192320-29290-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439250192320-29290-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439250192320-29290-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439250192320-29290-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.154.189:33105
--webserver_interface=127.28.154.189
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.189.keytab
--principal=kudu/127.28.154.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:34593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.154.190:44501,127.28.154.189:33105,127.28.154.188:38397
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 09:11:10.311281 21980 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.154.189" port: 33105 } attempt: 1
I20260811 09:11:10.312559 21986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:10.312039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.189:33105 (local address 127.0.0.1:59994)
0811 09:11:10.312256 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:10.312421 (+ 165us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.189:33105: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260811 09:11:10.313624 21986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:10.313268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.189:33105 (local address 127.0.0.1:60010)
0811 09:11:10.313413 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:10.313488 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.189:33105: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260811 09:11:10.313807 21980 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.154.189:33105: Network error: Client connection negotiation failed: client connection to 127.28.154.189:33105: connect: Connection refused (error 111)
I20260811 09:11:10.382079 21980 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.154.189" port: 33105 } attempt: 2
I20260811 09:11:10.383355 21986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:10.382855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.189:33105 (local address 127.0.0.1:60018)
0811 09:11:10.383121 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:10.383213 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.189:33105: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260811 09:11:10.384548 21986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:10.384081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.189:33105 (local address 127.0.0.1:60022)
0811 09:11:10.384262 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:10.384381 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.189:33105: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260811 09:11:10.384757 21980 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.154.189:33105: Network error: Client connection negotiation failed: client connection to 127.28.154.189:33105: connect: Connection refused (error 111)
I20260811 09:11:10.476032 21980 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.154.189" port: 33105 } attempt: 3
I20260811 09:11:10.477276 21986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:10.476819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.189:33105 (local address 127.0.0.1:60036)
0811 09:11:10.477030 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:10.477139 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.189:33105: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260811 09:11:10.478314 21986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:10.477946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.189:33105 (local address 127.0.0.1:60038)
0811 09:11:10.478103 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:10.478172 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.189:33105: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260811 09:11:10.478521 21980 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.154.189:33105: Network error: Client connection negotiation failed: client connection to 127.28.154.189:33105: connect: Connection refused (error 111)
I20260811 09:11:10.621760 21980 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.154.189" port: 33105 } attempt: 4
I20260811 09:11:10.622920 21986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:10.622389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.189:33105 (local address 127.0.0.1:60052)
0811 09:11:10.622689 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:10.622777 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.189:33105: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142}
I20260811 09:11:10.624048 21986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:10.623646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.189:33105 (local address 127.0.0.1:60068)
0811 09:11:10.623814 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:10.623917 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.189:33105: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260811 09:11:10.624281 21980 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.154.189:33105: Network error: Client connection negotiation failed: client connection to 127.28.154.189:33105: connect: Connection refused (error 111)
W20260811 09:11:10.628733 22000 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:11:10.629025 22000 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:11:10.629163 22000 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:11:10.636678 22000 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:11:10.636792 22000 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:11:10.636850 22000 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:11:10.636929 22000 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:11:10.636994 22000 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:11:10.647095 22000 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:34593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439250192320-29290-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439250192320-29290-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.28.154.190:44501,127.28.154.189:33105,127.28.154.188:38397
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.189.keytab
--principal=kudu/127.28.154.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.154.189:33105
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439250192320-29290-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.28.154.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439250192320-29290-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:11:10.648528 22000 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:11:10.649825 22000 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:11:10.661796 22010 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:11:10.661827 22012 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:11:10.661758 22015 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:11:10.662643 22000 server_base.cc:1034] running on GCE node
I20260811 09:11:10.663378 22000 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:11:10.664752 22000 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:11:10.665957 22000 hybrid_clock.cc:648] HybridClock initialized: now 1786439470665896 us; error 52 us; skew 500 ppm
I20260811 09:11:10.666270 22000 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:11:10 dist-test-slave-xt4k krb5kdc[21837](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439470, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:11:10.669608 22000 init.cc:377] Logged in from keytab as kudu/127.28.154.189@KRBTEST.COM (short username kudu)
I20260811 09:11:10.671273 22000 webserver.cc:460] Webserver started at http://127.28.154.189:40771/ using document root <none> and password file <none>
I20260811 09:11:10.672009 22000 fs_manager.cc:359] Metadata directory not provided
I20260811 09:11:10.672118 22000 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:11:10.672436 22000 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:11:10.675869 22000 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439250192320-29290-0/minicluster-data/master-1/data/instance:
uuid: "5b3ea92d2ce64afdbce7d2182e00ca06"
format_stamp: "Formatted at 2026-08-11 09:11:10 on dist-test-slave-xt4k"
server_key: "cbf604da09c3d1c1cc825bd3c851a18c"
server_key_iv: "f404637a7549b9f635f8221961cb25ea"
server_key_version: "encryptionkey@0"
I20260811 09:11:10.676795 22000 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439250192320-29290-0/minicluster-data/master-1/wal/instance:
uuid: "5b3ea92d2ce64afdbce7d2182e00ca06"
format_stamp: "Formatted at 2026-08-11 09:11:10 on dist-test-slave-xt4k"
server_key: "cbf604da09c3d1c1cc825bd3c851a18c"
server_key_iv: "f404637a7549b9f635f8221961cb25ea"
server_key_version: "encryptionkey@0"
I20260811 09:11:10.682128 22000 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 09:11:10.685745 22029 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:11:10.687196 22000 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 09:11:10.687354 22000 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439250192320-29290-0/minicluster-data/master-1/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439250192320-29290-0/minicluster-data/master-1/wal
uuid: "5b3ea92d2ce64afdbce7d2182e00ca06"
format_stamp: "Formatted at 2026-08-11 09:11:10 on dist-test-slave-xt4k"
server_key: "cbf604da09c3d1c1cc825bd3c851a18c"
server_key_iv: "f404637a7549b9f635f8221961cb25ea"
server_key_version: "encryptionkey@0"
I20260811 09:11:10.687558 22000 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439250192320-29290-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439250192320-29290-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439250192320-29290-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:11:10.731454 22000 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:11:10.735690 22000 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:11:10.735976 22000 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:11:10.753974 22000 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.189:33105
I20260811 09:11:10.754027 22119 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.189:33105 every 8 connection(s)
I20260811 09:11:10.755366 22000 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439250192320-29290-0/minicluster-data/master-1/data/info.pb
I20260811 09:11:10.759311 22121 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:11:10.761956 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 22000
I20260811 09:11:10.762151 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439250192320-29290-0/minicluster-data/master-1/wal/instance
I20260811 09:11:10.762804 29290 external_mini_cluster.cc:1428] Setting key e1dc2ef023e9fbebe6a871f9e27b8ba6
I20260811 09:11:10.768062 22121 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.154.190" port: 44501 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 09:11:10 dist-test-slave-xt4k krb5kdc[21837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439470, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.189@KRBTEST.COM for kudu/127.28.154.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.28.154.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.154.188@KRBTEST.COM" created.
I20260811 09:11:10.793394 22127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:10.771466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43556 (local address 127.28.154.190:44501)
0811 09:11:10.775592 (+ 4126us) server_negotiation.cc:206] Beginning negotiation
0811 09:11:10.775597 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:11:10.775618 (+ 21us) server_negotiation.cc:407] Connection header received
0811 09:11:10.777187 (+ 1569us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:11:10.777193 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:11:10.777283 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:11:10.777424 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:11:10.779015 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:10.780042 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:10.781671 (+ 1629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:10.782220 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:10.782348 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:10.786054 (+ 3706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:11:10.786063 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:11:10.786068 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:11:10.786136 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:11:10.789651 (+ 3515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:11:10.790543 (+ 892us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:11:10.790549 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:11:10.790552 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:11:10.790696 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:11:10.791357 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:11:10.791363 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:11:10.791367 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:11:10.791707 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:11:10.791848 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0811 09:11:10.792832 (+ 984us) server_negotiation.cc:299] Negotiation successful
0811 09:11:10.793109 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4023,"thread_start_us":3891,"threads_started":1}
I20260811 09:11:10.794606 22129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:10.772234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:44501 (local address 127.0.0.1:43556)
0811 09:11:10.774788 (+ 2554us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:10.774842 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 09:11:10.776881 (+ 2039us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:11:10.777656 (+ 775us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:11:10.777664 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:11:10.778202 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:11:10.778810 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:11:10.778829 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:11:10.780192 (+ 1363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:11:10.780196 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:11:10.781481 (+ 1285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:11:10.781496 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:11:10.782432 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:11:10.782437 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:11:10.782680 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:10.783363 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:11:10.783387 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:11:10.785755 (+ 2368us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:11:10.789842 (+ 4087us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:11:10.789851 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:11:10.789868 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:11:10.790364 (+ 496us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:11:10.790838 (+ 474us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:11:10.790843 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:11:10.790845 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:11:10.791104 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:11:10.791863 (+ 759us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:11:10.791871 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:11:10.792321 (+ 450us) client_negotiation.cc:769] Sending connection context
0811 09:11:10.792685 (+ 364us) client_negotiation.cc:240] Negotiation successful
0811 09:11:10.793194 (+ 509us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2217,"thread_start_us":1923,"threads_started":1}
I20260811 09:11:10.796691 22121 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.154.189" port: 33105 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 09:11:10 dist-test-slave-xt4k krb5kdc[21837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439470, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.189@KRBTEST.COM for kudu/127.28.154.189@KRBTEST.COM
I20260811 09:11:10.813424 22129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:10.797606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.189:33105 (local address 127.0.0.1:60084)
0811 09:11:10.797905 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:10.797926 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 09:11:10.798038 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:11:10.799194 (+ 1156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:11:10.799199 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:11:10.799718 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:11:10.799903 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:11:10.799915 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:11:10.801933 (+ 2018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:11:10.801939 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:11:10.803324 (+ 1385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:11:10.803338 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:11:10.804304 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:11:10.804309 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:11:10.804627 (+ 318us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:10.805274 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:11:10.805296 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:11:10.807801 (+ 2505us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:11:10.810945 (+ 3144us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:11:10.810951 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:11:10.810955 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:11:10.811313 (+ 358us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:11:10.811671 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:11:10.811677 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:11:10.811681 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:11:10.811829 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:11:10.812457 (+ 628us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:11:10.812463 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:11:10.812606 (+ 143us) client_negotiation.cc:769] Sending connection context
0811 09:11:10.812729 (+ 123us) client_negotiation.cc:240] Negotiation successful
0811 09:11:10.813005 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182}
I20260811 09:11:10.813465 22132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:10.797803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60084 (local address 127.28.154.189:33105)
0811 09:11:10.798623 (+ 820us) server_negotiation.cc:206] Beginning negotiation
0811 09:11:10.798632 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 09:11:10.798665 (+ 33us) server_negotiation.cc:407] Connection header received
0811 09:11:10.798722 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:11:10.798730 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:11:10.798980 (+ 250us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:11:10.799138 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:11:10.800127 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:10.801729 (+ 1602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:10.803455 (+ 1726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:10.804145 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:10.804269 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:10.808004 (+ 3735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:11:10.808016 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:11:10.808034 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:11:10.808069 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:11:10.810809 (+ 2740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:11:10.811438 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:11:10.811445 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:11:10.811447 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:11:10.811568 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:11:10.811928 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:11:10.811932 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:11:10.811934 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:11:10.812298 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:11:10.812549 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0811 09:11:10.812894 (+ 345us) server_negotiation.cc:299] Negotiation successful
0811 09:11:10.813108 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":616,"thread_start_us":438,"threads_started":1}
I20260811 09:11:10.816551 22121 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.154.188" port: 38397 } has no permanent_uuid. Determining permanent_uuid...
I20260811 09:11:10.817741 22129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:10.817251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.188:38397 (local address 127.0.0.1:37178)
0811 09:11:10.817409 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:10.817570 (+ 161us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.188:38397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260811 09:11:10.817966 22032 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.28.154.188:38397: connect: Connection refused (error 111)
I20260811 09:11:10.819530 22129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:10.819169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.188:38397 (local address 127.0.0.1:37186)
0811 09:11:10.819345 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:10.819419 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.188:38397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260811 09:11:10.819725 22121 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.154.188:38397: Network error: Client connection negotiation failed: client connection to 127.28.154.188:38397: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.154.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.188.keytab.
Entry for principal kudu/127.28.154.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.188.keytab.
I20260811 09:11:10.830147 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439250192320-29290-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439250192320-29290-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439250192320-29290-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439250192320-29290-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.154.188:38397
--webserver_interface=127.28.154.188
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.188.keytab
--principal=kudu/127.28.154.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:34593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.154.190:44501,127.28.154.189:33105,127.28.154.188:38397
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 09:11:10.869047 22121 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.154.188" port: 38397 } attempt: 1
I20260811 09:11:10.870440 22129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:10.869795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.188:38397 (local address 127.0.0.1:37200)
0811 09:11:10.870037 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:10.870149 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.188:38397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
I20260811 09:11:10.871654 22129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:10.871231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.188:38397 (local address 127.0.0.1:37214)
0811 09:11:10.871404 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:10.871528 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.188:38397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260811 09:11:10.871876 22121 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.154.188:38397: Network error: Client connection negotiation failed: client connection to 127.28.154.188:38397: connect: Connection refused (error 111)
I20260811 09:11:10.923614 21980 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.154.189" port: 33105 } attempt: 5
Aug 11 09:11:10 dist-test-slave-xt4k krb5kdc[21837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439470, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.190@KRBTEST.COM for kudu/127.28.154.189@KRBTEST.COM
I20260811 09:11:10.940125 22121 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.154.188" port: 38397 } attempt: 2
I20260811 09:11:10.943426 21986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:10.924447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.189:33105 (local address 127.0.0.1:60086)
0811 09:11:10.924697 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:10.924719 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 09:11:10.924854 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:11:10.925264 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:11:10.925269 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:11:10.925791 (+ 522us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:11:10.926036 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:11:10.926051 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:11:10.927142 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:11:10.927148 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:11:10.929083 (+ 1935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:11:10.929105 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:11:10.930079 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:11:10.930086 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:11:10.930335 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:10.931056 (+ 721us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:11:10.931081 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:11:10.933823 (+ 2742us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:11:10.940069 (+ 6246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:11:10.940076 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:11:10.940081 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:11:10.940665 (+ 584us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:11:10.941181 (+ 516us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:11:10.941187 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:11:10.941191 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:11:10.941380 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:11:10.942117 (+ 737us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:11:10.942122 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:11:10.942380 (+ 258us) client_negotiation.cc:769] Sending connection context
0811 09:11:10.942558 (+ 178us) client_negotiation.cc:240] Negotiation successful
0811 09:11:10.943063 (+ 505us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100}
I20260811 09:11:10.943533 22132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:10.924764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60086 (local address 127.28.154.189:33105)
0811 09:11:10.924938 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0811 09:11:10.924943 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:11:10.924962 (+ 19us) server_negotiation.cc:407] Connection header received
0811 09:11:10.925027 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:11:10.925031 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:11:10.925112 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:11:10.925192 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:11:10.926253 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:10.926978 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:10.929304 (+ 2326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:10.929819 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:10.929935 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:10.934068 (+ 4133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:11:10.934074 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:11:10.934078 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:11:10.934115 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:11:10.939835 (+ 5720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:11:10.940871 (+ 1036us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:11:10.940878 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:11:10.940882 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:11:10.941048 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:11:10.941527 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:11:10.941547 (+ 20us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:11:10.941551 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:11:10.941942 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:11:10.942112 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0811 09:11:10.942771 (+ 659us) server_negotiation.cc:299] Negotiation successful
0811 09:11:10.943130 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 09:11:10.945382 22129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:10.944918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.188:38397 (local address 127.0.0.1:37224)
0811 09:11:10.945108 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:10.945209 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.188:38397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260811 09:11:10.949105 21980 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.154.188" port: 38397 } has no permanent_uuid. Determining permanent_uuid...
I20260811 09:11:10.949913 22129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:10.949230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.188:38397 (local address 127.0.0.1:37228)
0811 09:11:10.949565 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:10.949674 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.188:38397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
W20260811 09:11:10.950201 22121 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.154.188:38397: Network error: Client connection negotiation failed: client connection to 127.28.154.188:38397: connect: Connection refused (error 111)
I20260811 09:11:10.950608 21986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:10.950025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.188:38397 (local address 127.0.0.1:37230)
0811 09:11:10.950295 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:10.950398 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.188:38397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153}
I20260811 09:11:10.951897 21986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:10.951495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.188:38397 (local address 127.0.0.1:37238)
0811 09:11:10.951670 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:10.951765 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.188:38397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260811 09:11:10.952165 21980 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.154.188:38397: Network error: Client connection negotiation failed: client connection to 127.28.154.188:38397: connect: Connection refused (error 111)
I20260811 09:11:11.003463 21980 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.154.188" port: 38397 } attempt: 1
I20260811 09:11:11.005008 21986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:11.004485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.188:38397 (local address 127.0.0.1:37248)
0811 09:11:11.004713 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:11.004808 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.188:38397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260811 09:11:11.006170 21986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:11.005765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.188:38397 (local address 127.0.0.1:37250)
0811 09:11:11.005967 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:11.006046 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.188:38397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
W20260811 09:11:11.006394 21980 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.154.188:38397: Network error: Client connection negotiation failed: client connection to 127.28.154.188:38397: connect: Connection refused (error 111)
I20260811 09:11:11.041479 22121 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.154.188" port: 38397 } attempt: 3
I20260811 09:11:11.042886 22129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:11.042257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.188:38397 (local address 127.0.0.1:37262)
0811 09:11:11.042533 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:11.042670 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.188:38397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
I20260811 09:11:11.044219 22129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:11.043642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.188:38397 (local address 127.0.0.1:37268)
0811 09:11:11.043892 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:11.044011 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.188:38397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139}
W20260811 09:11:11.044550 22121 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.154.188:38397: Network error: Client connection negotiation failed: client connection to 127.28.154.188:38397: connect: Connection refused (error 111)
I20260811 09:11:11.074627 21980 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.154.188" port: 38397 } attempt: 2
I20260811 09:11:11.075856 21986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:11.075357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.188:38397 (local address 127.0.0.1:37284)
0811 09:11:11.075600 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:11.075693 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.188:38397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260811 09:11:11.076925 21986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:11.076546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.188:38397 (local address 127.0.0.1:37292)
0811 09:11:11.076728 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:11.076805 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.188:38397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260811 09:11:11.077157 21980 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.154.188:38397: Network error: Client connection negotiation failed: client connection to 127.28.154.188:38397: connect: Connection refused (error 111)
I20260811 09:11:11.183391 21980 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.154.188" port: 38397 } attempt: 3
I20260811 09:11:11.184721 21986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:11.184112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.188:38397 (local address 127.0.0.1:37300)
0811 09:11:11.184320 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:11.184482 (+ 162us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.188:38397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260811 09:11:11.185724 21986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:11.185389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.188:38397 (local address 127.0.0.1:37310)
0811 09:11:11.185548 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:11.185620 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.188:38397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260811 09:11:11.185936 21980 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.154.188:38397: Network error: Client connection negotiation failed: client connection to 127.28.154.188:38397: connect: Connection refused (error 111)
I20260811 09:11:11.187811 22121 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.154.188" port: 38397 } attempt: 4
I20260811 09:11:11.188910 22129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:11.188458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.188:38397 (local address 127.0.0.1:37312)
0811 09:11:11.188628 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:11.188724 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.188:38397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260811 09:11:11.190086 22129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:11.189711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.188:38397 (local address 127.0.0.1:37328)
0811 09:11:11.189863 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:11.189959 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.188:38397: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260811 09:11:11.190342 22121 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.154.188:38397: Network error: Client connection negotiation failed: client connection to 127.28.154.188:38397: connect: Connection refused (error 111)
W20260811 09:11:11.196216 22137 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:11:11.196557 22137 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:11:11.196691 22137 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:11:11.204144 22137 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:11:11.204255 22137 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:11:11.204303 22137 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:11:11.204393 22137 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:11:11.204465 22137 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:11:11.214541 22137 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:34593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439250192320-29290-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439250192320-29290-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.28.154.190:44501,127.28.154.189:33105,127.28.154.188:38397
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.188.keytab
--principal=kudu/127.28.154.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.154.188:38397
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439250192320-29290-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.28.154.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439250192320-29290-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:11:11.215996 22137 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:11:11.217394 22137 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:11:11.229928 22151 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:11:11.230057 22155 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:11:11.230007 22160 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:11:11.231109 22137 server_base.cc:1034] running on GCE node
I20260811 09:11:11.231983 22137 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:11:11.233560 22137 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:11:11.234786 22137 hybrid_clock.cc:648] HybridClock initialized: now 1786439471234724 us; error 50 us; skew 500 ppm
I20260811 09:11:11.235136 22137 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:11:11 dist-test-slave-xt4k krb5kdc[21837](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439471, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:11:11.243541 22137 init.cc:377] Logged in from keytab as kudu/127.28.154.188@KRBTEST.COM (short username kudu)
I20260811 09:11:11.245344 22137 webserver.cc:460] Webserver started at http://127.28.154.188:45731/ using document root <none> and password file <none>
I20260811 09:11:11.246088 22137 fs_manager.cc:359] Metadata directory not provided
I20260811 09:11:11.246215 22137 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:11:11.246524 22137 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:11:11.249953 22137 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439250192320-29290-0/minicluster-data/master-2/data/instance:
uuid: "802c96530cd3461b8f50680cc4b69eee"
format_stamp: "Formatted at 2026-08-11 09:11:11 on dist-test-slave-xt4k"
server_key: "74e3911aafb1aff4ee530aa554c4380f"
server_key_iv: "a789f169e053186ad99bfefc87b3dc7f"
server_key_version: "encryptionkey@0"
I20260811 09:11:11.250762 22137 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439250192320-29290-0/minicluster-data/master-2/wal/instance:
uuid: "802c96530cd3461b8f50680cc4b69eee"
format_stamp: "Formatted at 2026-08-11 09:11:11 on dist-test-slave-xt4k"
server_key: "74e3911aafb1aff4ee530aa554c4380f"
server_key_iv: "a789f169e053186ad99bfefc87b3dc7f"
server_key_version: "encryptionkey@0"
I20260811 09:11:11.256093 22137 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 09:11:11.259828 22171 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:11:11.261312 22137 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 09:11:11.261526 22137 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439250192320-29290-0/minicluster-data/master-2/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439250192320-29290-0/minicluster-data/master-2/wal
uuid: "802c96530cd3461b8f50680cc4b69eee"
format_stamp: "Formatted at 2026-08-11 09:11:11 on dist-test-slave-xt4k"
server_key: "74e3911aafb1aff4ee530aa554c4380f"
server_key_iv: "a789f169e053186ad99bfefc87b3dc7f"
server_key_version: "encryptionkey@0"
I20260811 09:11:11.261722 22137 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439250192320-29290-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439250192320-29290-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439250192320-29290-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:11:11.296101 22137 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:11:11.301344 22137 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:11:11.301659 22137 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:11:11.321269 22137 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.188:38397
I20260811 09:11:11.321311 22269 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.188:38397 every 8 connection(s)
I20260811 09:11:11.322762 22137 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439250192320-29290-0/minicluster-data/master-2/data/info.pb
I20260811 09:11:11.326648 22270 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:11:11.330138 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 22137
I20260811 09:11:11.330323 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786439250192320-29290-0/minicluster-data/master-2/wal/instance
I20260811 09:11:11.330843 29290 external_mini_cluster.cc:1428] Setting key 5ec9bb30859b85dec479208f7eee1225
I20260811 09:11:11.334350 22270 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.154.190" port: 44501 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 09:11:11 dist-test-slave-xt4k krb5kdc[21837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439469, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.154.188@KRBTEST.COM
Aug 11 09:11:11 dist-test-slave-xt4k krb5kdc[21837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439471, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.188@KRBTEST.COM for kudu/127.28.154.190@KRBTEST.COM
I20260811 09:11:11.355226 22278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:11.338906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43564 (local address 127.28.154.190:44501)
0811 09:11:11.340147 (+ 1241us) server_negotiation.cc:206] Beginning negotiation
0811 09:11:11.340152 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:11:11.340167 (+ 15us) server_negotiation.cc:407] Connection header received
0811 09:11:11.340216 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:11:11.340220 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:11:11.340290 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:11:11.340417 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:11:11.341580 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:11.342290 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:11.343693 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:11.344253 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:11.344424 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:11.348085 (+ 3661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:11:11.348093 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:11:11.348098 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:11:11.348151 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:11:11.351680 (+ 3529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:11:11.352782 (+ 1102us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:11:11.352787 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:11:11.352790 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:11:11.352929 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:11:11.353346 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:11:11.353350 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:11:11.353352 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:11:11.353649 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:11:11.353766 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 09:11:11.354558 (+ 792us) server_negotiation.cc:299] Negotiation successful
0811 09:11:11.354790 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1139,"thread_start_us":1046,"threads_started":1}
I20260811 09:11:11.356521 22277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:11.338746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:44501 (local address 127.0.0.1:43564)
0811 09:11:11.339594 (+ 848us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:11.339639 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 09:11:11.339801 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:11:11.340507 (+ 706us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:11:11.340514 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:11:11.341153 (+ 639us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:11:11.341405 (+ 252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:11:11.341415 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:11:11.342463 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:11:11.342467 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:11:11.343543 (+ 1076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:11:11.343556 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:11:11.344533 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:11:11.344539 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:11:11.344667 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:11.345257 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:11:11.345281 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:11:11.347853 (+ 2572us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:11:11.351889 (+ 4036us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:11:11.351903 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:11:11.351918 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:11:11.352604 (+ 686us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:11:11.353067 (+ 463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:11:11.353071 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:11:11.353073 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:11:11.353230 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:11:11.353794 (+ 564us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:11:11.353802 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:11:11.354109 (+ 307us) client_negotiation.cc:769] Sending connection context
0811 09:11:11.354426 (+ 317us) client_negotiation.cc:240] Negotiation successful
0811 09:11:11.354935 (+ 509us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":596,"mutex_wait_us":53,"thread_start_us":405,"threads_started":1}
I20260811 09:11:11.356539 22276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:11.332854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37338 (local address 127.28.154.188:38397)
0811 09:11:11.333703 (+ 849us) server_negotiation.cc:206] Beginning negotiation
0811 09:11:11.333719 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 09:11:11.333773 (+ 54us) server_negotiation.cc:407] Connection header received
0811 09:11:11.335645 (+ 1872us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:11:11.335679 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:11:11.336681 (+ 1002us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:11:11.337156 (+ 475us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:11:11.338903 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:11.340747 (+ 1844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:11.342222 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:11.342945 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:11.343120 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:11.347714 (+ 4594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:11:11.347724 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:11:11.347739 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:11:11.347775 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:11:11.351598 (+ 3823us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:11:11.352760 (+ 1162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:11:11.352767 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:11:11.352770 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:11:11.352939 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:11:11.353334 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:11:11.353339 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:11:11.353342 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:11:11.354067 (+ 725us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:11:11.354365 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0811 09:11:11.354676 (+ 311us) server_negotiation.cc:299] Negotiation successful
0811 09:11:11.355067 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":156,"server-negotiator.queue_time_us":586,"thread_start_us":340,"threads_started":1}
I20260811 09:11:11.358386 22270 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.154.189" port: 33105 } has no permanent_uuid. Determining permanent_uuid...
I20260811 09:11:11.363175 21980 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.154.188" port: 38397 } attempt: 4
Aug 11 09:11:11 dist-test-slave-xt4k krb5kdc[21837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439471, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.188@KRBTEST.COM for kudu/127.28.154.189@KRBTEST.COM
Aug 11 09:11:11 dist-test-slave-xt4k krb5kdc[21837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439469, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.154.190@KRBTEST.COM
Aug 11 09:11:11 dist-test-slave-xt4k krb5kdc[21837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439470, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.190@KRBTEST.COM for kudu/127.28.154.188@KRBTEST.COM
I20260811 09:11:11.374600 22132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:11.359161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60096 (local address 127.28.154.189:33105)
0811 09:11:11.359387 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0811 09:11:11.359394 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:11:11.359444 (+ 50us) server_negotiation.cc:407] Connection header received
0811 09:11:11.359690 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:11:11.359695 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:11:11.359781 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:11:11.359914 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:11:11.360971 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:11.362042 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:11.363583 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:11.364135 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:11.364256 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:11.368017 (+ 3761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:11:11.368023 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:11:11.368026 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:11:11.368062 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:11:11.371107 (+ 3045us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:11:11.371873 (+ 766us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:11:11.371878 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:11:11.371880 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:11:11.372028 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:11:11.372504 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:11:11.372510 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:11:11.372513 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:11:11.372820 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:11:11.372976 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0811 09:11:11.373743 (+ 767us) server_negotiation.cc:299] Negotiation successful
0811 09:11:11.374087 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 09:11:11.374861 22277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:11.359224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.189:33105 (local address 127.0.0.1:60096)
0811 09:11:11.359371 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:11.359390 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 09:11:11.359513 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:11:11.360006 (+ 493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:11:11.360011 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:11:11.360509 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:11:11.360812 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:11:11.360826 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:11:11.362217 (+ 1391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:11:11.362221 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:11:11.363440 (+ 1219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:11:11.363455 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:11:11.364281 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:11:11.364286 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:11:11.364531 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:11.365099 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:11:11.365117 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:11:11.367802 (+ 2685us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:11:11.371244 (+ 3442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:11:11.371249 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:11:11.371254 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:11:11.371755 (+ 501us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:11:11.372162 (+ 407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:11:11.372165 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:11:11.372168 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:11:11.372309 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:11:11.373051 (+ 742us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:11:11.373056 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:11:11.373253 (+ 197us) client_negotiation.cc:769] Sending connection context
0811 09:11:11.373425 (+ 172us) client_negotiation.cc:240] Negotiation successful
0811 09:11:11.374246 (+ 821us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260811 09:11:11.376461 22270 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.154.188" port: 38397 } has no permanent_uuid. Determining permanent_uuid...
I20260811 09:11:11.379964 22278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:11.360267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43578 (local address 127.28.154.190:44501)
0811 09:11:11.360493 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0811 09:11:11.360498 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:11:11.360515 (+ 17us) server_negotiation.cc:407] Connection header received
0811 09:11:11.360592 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:11:11.360595 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:11:11.360665 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:11:11.360747 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:11:11.362366 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:11.363039 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:11.365221 (+ 2182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:11.366024 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:11.366227 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:11.372839 (+ 6612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:11:11.372846 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:11:11.372850 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:11:11.372889 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:11:11.376166 (+ 3277us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:11:11.377074 (+ 908us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:11:11.377079 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:11:11.377082 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:11:11.377255 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:11:11.377735 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:11:11.377741 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:11:11.377745 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:11:11.378137 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:11:11.378426 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0811 09:11:11.379511 (+ 1085us) server_negotiation.cc:299] Negotiation successful
0811 09:11:11.379726 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":90}
I20260811 09:11:11.379998 21986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:11.363757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.188:38397 (local address 127.0.0.1:37342)
0811 09:11:11.363954 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:11.363973 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 09:11:11.364074 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:11:11.364748 (+ 674us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:11:11.364752 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:11:11.365290 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:11:11.365474 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:11:11.365487 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:11:11.366897 (+ 1410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:11:11.366902 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:11:11.367900 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:11:11.367913 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:11:11.368844 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:11:11.368851 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:11:11.368996 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:11.369515 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:11:11.369540 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:11:11.372401 (+ 2861us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:11:11.376792 (+ 4391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:11:11.376798 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:11:11.376803 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:11:11.377288 (+ 485us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:11:11.378248 (+ 960us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:11:11.378253 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:11:11.378257 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:11:11.378466 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:11:11.378995 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:11:11.379001 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:11:11.379164 (+ 163us) client_negotiation.cc:769] Sending connection context
0811 09:11:11.379316 (+ 152us) client_negotiation.cc:240] Negotiation successful
0811 09:11:11.379599 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":69}
I20260811 09:11:11.381357 22276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:11.364086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37342 (local address 127.28.154.188:38397)
0811 09:11:11.364487 (+ 401us) server_negotiation.cc:206] Beginning negotiation
0811 09:11:11.364492 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:11:11.364507 (+ 15us) server_negotiation.cc:407] Connection header received
0811 09:11:11.364545 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:11:11.364548 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:11:11.364602 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:11:11.364697 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:11:11.365638 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:11.366771 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:11.368054 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:11.368677 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:11.368803 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:11.372630 (+ 3827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:11:11.372637 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:11:11.372641 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:11:11.372677 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:11:11.376609 (+ 3932us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:11:11.377918 (+ 1309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:11:11.377923 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:11:11.377926 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:11:11.378099 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:11:11.378562 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:11:11.378568 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:11:11.378571 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:11:11.378849 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:11:11.378978 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0811 09:11:11.380428 (+ 1450us) server_negotiation.cc:299] Negotiation successful
0811 09:11:11.381052 (+ 624us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289}
Aug 11 09:11:11 dist-test-slave-xt4k krb5kdc[21837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439471, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.188@KRBTEST.COM for kudu/127.28.154.188@KRBTEST.COM
I20260811 09:11:11.387462 21980 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8d3967ffc3e4bda924776881f306353: Bootstrap starting.
I20260811 09:11:11.391222 21980 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e8d3967ffc3e4bda924776881f306353: Neither blocks nor log segments found. Creating new log.
Aug 11 09:11:11 dist-test-slave-xt4k krb5kdc[21837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439469, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.154.189@KRBTEST.COM
I20260811 09:11:11.392642 21980 log.cc:824] T 00000000000000000000000000000000 P e8d3967ffc3e4bda924776881f306353: Log is configured to *not* fsync() on all Append() calls
I20260811 09:11:11.393244 22277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:11.377238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.188:38397 (local address 127.0.0.1:37352)
0811 09:11:11.377412 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:11.377433 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 09:11:11.377545 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:11:11.378272 (+ 727us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:11:11.378276 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:11:11.378764 (+ 488us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:11:11.378957 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:11:11.378969 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:11:11.380548 (+ 1579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:11:11.380554 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:11:11.381837 (+ 1283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:11:11.381851 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:11:11.382761 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:11:11.382767 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:11:11.382948 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:11.383451 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:11:11.383466 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:11:11.385762 (+ 2296us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:11:11.390160 (+ 4398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:11:11.390166 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:11:11.390171 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:11:11.390680 (+ 509us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:11:11.391121 (+ 441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:11:11.391126 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:11:11.391130 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:11:11.391301 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:11:11.392014 (+ 713us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:11:11.392020 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:11:11.392211 (+ 191us) client_negotiation.cc:769] Sending connection context
0811 09:11:11.392455 (+ 244us) client_negotiation.cc:240] Negotiation successful
0811 09:11:11.392829 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260811 09:11:11.393679 22279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:11.377502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37352 (local address 127.28.154.188:38397)
0811 09:11:11.378041 (+ 539us) server_negotiation.cc:206] Beginning negotiation
0811 09:11:11.378045 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:11:11.378063 (+ 18us) server_negotiation.cc:407] Connection header received
0811 09:11:11.378099 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:11:11.378102 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:11:11.378152 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:11:11.378252 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:11:11.379095 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:11.380373 (+ 1278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:11.381995 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:11.382631 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:11.382739 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:11.385984 (+ 3245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:11:11.385990 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:11:11.385995 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:11:11.386031 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:11:11.389996 (+ 3965us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:11:11.390820 (+ 824us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:11:11.390826 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:11:11.390829 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:11:11.390992 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:11:11.391430 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:11:11.391434 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:11:11.391437 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:11:11.391863 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:11:11.392140 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0811 09:11:11.392598 (+ 458us) server_negotiation.cc:299] Negotiation successful
0811 09:11:11.393011 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":174,"server-negotiator.queue_time_us":436,"thread_start_us":275,"threads_started":1}
I20260811 09:11:11.396749 21980 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8d3967ffc3e4bda924776881f306353: No bootstrap required, opened a new log
I20260811 09:11:11.398344 22270 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 802c96530cd3461b8f50680cc4b69eee: Bootstrap starting.
I20260811 09:11:11.400929 22132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:11.382022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60112 (local address 127.28.154.189:33105)
0811 09:11:11.382199 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0811 09:11:11.382205 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:11:11.382222 (+ 17us) server_negotiation.cc:407] Connection header received
0811 09:11:11.382273 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:11:11.382279 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:11:11.382350 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:11:11.382462 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:11:11.384141 (+ 1679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:11.384993 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:11.386447 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:11.387082 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:11.387217 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:11.393038 (+ 5821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:11:11.393045 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:11:11.393051 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:11:11.393095 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:11:11.396817 (+ 3722us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:11:11.397782 (+ 965us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:11:11.397787 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:11:11.397791 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:11:11.397978 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:11:11.398445 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:11:11.398450 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:11:11.398453 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:11:11.398790 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:11:11.399082 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0811 09:11:11.400327 (+ 1245us) server_negotiation.cc:299] Negotiation successful
0811 09:11:11.400588 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 09:11:11.402141 22270 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 802c96530cd3461b8f50680cc4b69eee: Neither blocks nor log segments found. Creating new log.
I20260811 09:11:11.403591 22270 log.cc:824] T 00000000000000000000000000000000 P 802c96530cd3461b8f50680cc4b69eee: Log is configured to *not* fsync() on all Append() calls
I20260811 09:11:11.405993 29290 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 09:11:11.407317 21980 raft_consensus.cc:348] T 00000000000000000000000000000000 P e8d3967ffc3e4bda924776881f306353 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8d3967ffc3e4bda924776881f306353" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 44501 } } peers { permanent_uuid: "5b3ea92d2ce64afdbce7d2182e00ca06" member_type: VOTER last_known_addr { host: "127.28.154.189" port: 33105 } } peers { permanent_uuid: "802c96530cd3461b8f50680cc4b69eee" member_type: VOTER last_known_addr { host: "127.28.154.188" port: 38397 } }
I20260811 09:11:11.407712 21980 raft_consensus.cc:374] T 00000000000000000000000000000000 P e8d3967ffc3e4bda924776881f306353 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:11:11.407806 21980 raft_consensus.cc:729] T 00000000000000000000000000000000 P e8d3967ffc3e4bda924776881f306353 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8d3967ffc3e4bda924776881f306353, State: Initialized, Role: FOLLOWER
I20260811 09:11:11.408601 21980 consensus_queue.cc:260] T 00000000000000000000000000000000 P e8d3967ffc3e4bda924776881f306353 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8d3967ffc3e4bda924776881f306353" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 44501 } } peers { permanent_uuid: "5b3ea92d2ce64afdbce7d2182e00ca06" member_type: VOTER last_known_addr { host: "127.28.154.189" port: 33105 } } peers { permanent_uuid: "802c96530cd3461b8f50680cc4b69eee" member_type: VOTER last_known_addr { host: "127.28.154.188" port: 38397 } }
I20260811 09:11:11.413781 22282 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8d3967ffc3e4bda924776881f306353 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8d3967ffc3e4bda924776881f306353" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 44501 } } peers { permanent_uuid: "5b3ea92d2ce64afdbce7d2182e00ca06" member_type: VOTER last_known_addr { host: "127.28.154.189" port: 33105 } } peers { permanent_uuid: "802c96530cd3461b8f50680cc4b69eee" member_type: VOTER last_known_addr { host: "127.28.154.188" port: 38397 } } }
I20260811 09:11:11.414955 22270 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 802c96530cd3461b8f50680cc4b69eee: No bootstrap required, opened a new log
I20260811 09:11:11.416621 22282 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8d3967ffc3e4bda924776881f306353 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 09:11:11.416146 21980 sys_catalog.cc:564] T 00000000000000000000000000000000 P e8d3967ffc3e4bda924776881f306353 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:11:11.427116 22270 raft_consensus.cc:348] T 00000000000000000000000000000000 P 802c96530cd3461b8f50680cc4b69eee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8d3967ffc3e4bda924776881f306353" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 44501 } } peers { permanent_uuid: "5b3ea92d2ce64afdbce7d2182e00ca06" member_type: VOTER last_known_addr { host: "127.28.154.189" port: 33105 } } peers { permanent_uuid: "802c96530cd3461b8f50680cc4b69eee" member_type: VOTER last_known_addr { host: "127.28.154.188" port: 38397 } }
I20260811 09:11:11.427634 22270 raft_consensus.cc:374] T 00000000000000000000000000000000 P 802c96530cd3461b8f50680cc4b69eee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:11:11.427760 22270 raft_consensus.cc:729] T 00000000000000000000000000000000 P 802c96530cd3461b8f50680cc4b69eee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 802c96530cd3461b8f50680cc4b69eee, State: Initialized, Role: FOLLOWER
I20260811 09:11:11.428735 22270 consensus_queue.cc:260] T 00000000000000000000000000000000 P 802c96530cd3461b8f50680cc4b69eee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8d3967ffc3e4bda924776881f306353" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 44501 } } peers { permanent_uuid: "5b3ea92d2ce64afdbce7d2182e00ca06" member_type: VOTER last_known_addr { host: "127.28.154.189" port: 33105 } } peers { permanent_uuid: "802c96530cd3461b8f50680cc4b69eee" member_type: VOTER last_known_addr { host: "127.28.154.188" port: 38397 } }
I20260811 09:11:11.446339 22290 sys_catalog.cc:455] T 00000000000000000000000000000000 P 802c96530cd3461b8f50680cc4b69eee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8d3967ffc3e4bda924776881f306353" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 44501 } } peers { permanent_uuid: "5b3ea92d2ce64afdbce7d2182e00ca06" member_type: VOTER last_known_addr { host: "127.28.154.189" port: 33105 } } peers { permanent_uuid: "802c96530cd3461b8f50680cc4b69eee" member_type: VOTER last_known_addr { host: "127.28.154.188" port: 38397 } } }
I20260811 09:11:11.446839 22290 sys_catalog.cc:458] T 00000000000000000000000000000000 P 802c96530cd3461b8f50680cc4b69eee [sys.catalog]: This master's current role is: FOLLOWER
I20260811 09:11:11.449008 22132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:11.420094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60120 (local address 127.28.154.189:33105)
0811 09:11:11.420554 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0811 09:11:11.420560 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:11:11.433075 (+ 12515us) server_negotiation.cc:407] Connection header received
0811 09:11:11.433681 (+ 606us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:11:11.433687 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:11:11.433776 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:11:11.433875 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:11:11.435580 (+ 1705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:11.436315 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:11.437756 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:11.438185 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:11.438298 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:11.441585 (+ 3287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:11:11.441592 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:11:11.441597 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:11:11.441639 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:11:11.445290 (+ 3651us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:11:11.446421 (+ 1131us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:11:11.446427 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:11:11.446430 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:11:11.446612 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:11:11.447157 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:11:11.447161 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:11:11.447164 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:11:11.447526 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:11:11.447735 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0811 09:11:11.448429 (+ 694us) server_negotiation.cc:299] Negotiation successful
0811 09:11:11.448678 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311}
I20260811 09:11:11.449004 22270 sys_catalog.cc:564] T 00000000000000000000000000000000 P 802c96530cd3461b8f50680cc4b69eee [sys.catalog]: configured and running, proceeding with master startup.
W20260811 09:11:11.451426 21893 tablet.cc:2369] T 00000000000000000000000000000000 P e8d3967ffc3e4bda924776881f306353: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 09:11:11.462208 22278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:11.419824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43584 (local address 127.28.154.190:44501)
0811 09:11:11.420020 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 09:11:11.420027 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:11:11.429955 (+ 9928us) server_negotiation.cc:407] Connection header received
0811 09:11:11.430192 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:11:11.430199 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:11:11.430307 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:11:11.430417 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:11:11.441990 (+ 11573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:11.443261 (+ 1271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:11.448965 (+ 5704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:11.449721 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:11.449838 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:11.453802 (+ 3964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:11:11.453811 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:11:11.453816 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:11:11.453862 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:11:11.457544 (+ 3682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:11:11.458515 (+ 971us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:11:11.458522 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:11:11.458527 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:11:11.458738 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:11:11.459229 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:11:11.459235 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:11:11.459239 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:11:11.459872 (+ 633us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:11:11.460204 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0811 09:11:11.461474 (+ 1270us) server_negotiation.cc:299] Negotiation successful
0811 09:11:11.461753 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 09:11:11.463649 22279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:11.430807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37360 (local address 127.28.154.188:38397)
0811 09:11:11.431007 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 09:11:11.431013 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:11:11.434124 (+ 3111us) server_negotiation.cc:407] Connection header received
0811 09:11:11.434404 (+ 280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:11:11.434411 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:11:11.434513 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:11:11.434586 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:11:11.437134 (+ 2548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:11.438301 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:11.440872 (+ 2571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:11.441618 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:11.441726 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:11.452686 (+ 10960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:11:11.452695 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:11:11.452701 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:11:11.452751 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:11:11.459398 (+ 6647us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:11:11.460402 (+ 1004us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:11:11.460410 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:11:11.460413 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:11:11.460589 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:11:11.461077 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:11:11.461083 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:11:11.461087 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:11:11.461503 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:11:11.461774 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0811 09:11:11.462982 (+ 1208us) server_negotiation.cc:299] Negotiation successful
0811 09:11:11.463254 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
W20260811 09:11:11.483048 22311 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e8d3967ffc3e4bda924776881f306353: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 09:11:11.483210 22311 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 09:11:11.472867 21859 master_runner.cc:186] Error getting master registration for 127.28.154.189:33105: OK, instance_id { permanent_uuid: "5b3ea92d2ce64afdbce7d2182e00ca06" instance_seqno: 1786439470748623 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 09:11:11.483881 21859 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 09:11:11.489547 22121 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.154.188" port: 38397 } attempt: 5
I20260811 09:11:11.492130 22137 master_runner.cc:186] Error getting master registration for 127.28.154.189:33105: OK, instance_id { permanent_uuid: "5b3ea92d2ce64afdbce7d2182e00ca06" instance_seqno: 1786439470748623 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 09:11:11.503265 22137 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Aug 11 09:11:11 dist-test-slave-xt4k krb5kdc[21837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439470, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.189@KRBTEST.COM for kudu/127.28.154.188@KRBTEST.COM
W20260811 09:11:11.506932 22327 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 802c96530cd3461b8f50680cc4b69eee: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 09:11:11.507063 22327 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 09:11:11.512915 22129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:11.490083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.188:38397 (local address 127.0.0.1:37374)
0811 09:11:11.490310 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:11.490334 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 09:11:11.490456 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:11:11.491740 (+ 1284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:11:11.491746 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:11:11.492286 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:11:11.492529 (+ 243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:11:11.492542 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:11:11.494201 (+ 1659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:11:11.494207 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:11:11.495591 (+ 1384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:11:11.495609 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:11:11.496938 (+ 1329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:11:11.496944 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:11:11.497182 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:11.501542 (+ 4360us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:11:11.501567 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:11:11.504174 (+ 2607us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:11:11.508186 (+ 4012us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:11:11.508192 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:11:11.508196 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:11:11.508710 (+ 514us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:11:11.509175 (+ 465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:11:11.509180 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:11:11.509183 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:11:11.509343 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:11:11.511891 (+ 2548us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:11:11.511897 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:11:11.512064 (+ 167us) client_negotiation.cc:769] Sending connection context
0811 09:11:11.512184 (+ 120us) client_negotiation.cc:240] Negotiation successful
0811 09:11:11.512528 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":75}
I20260811 09:11:11.512957 22279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:11.491108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37374 (local address 127.28.154.188:38397)
0811 09:11:11.491273 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0811 09:11:11.491279 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:11:11.491294 (+ 15us) server_negotiation.cc:407] Connection header received
0811 09:11:11.491343 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:11:11.491348 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:11:11.491427 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:11:11.491552 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:11:11.492727 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:11.494029 (+ 1302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:11.495781 (+ 1752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:11.496548 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:11.496687 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:11.504430 (+ 7743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:11:11.504436 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:11:11.504441 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:11:11.504481 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:11:11.508024 (+ 3543us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:11:11.508876 (+ 852us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:11:11.508881 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:11:11.508883 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:11:11.509062 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:11:11.509510 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:11:11.509515 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:11:11.509518 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:11:11.511754 (+ 2236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:11:11.511922 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0811 09:11:11.512378 (+ 456us) server_negotiation.cc:299] Negotiation successful
0811 09:11:11.512607 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 09:11:11.517843 22121 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b3ea92d2ce64afdbce7d2182e00ca06: Bootstrap starting.
I20260811 09:11:11.522207 22121 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5b3ea92d2ce64afdbce7d2182e00ca06: Neither blocks nor log segments found. Creating new log.
I20260811 09:11:11.523296 22121 log.cc:824] T 00000000000000000000000000000000 P 5b3ea92d2ce64afdbce7d2182e00ca06: Log is configured to *not* fsync() on all Append() calls
I20260811 09:11:11.525969 22121 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b3ea92d2ce64afdbce7d2182e00ca06: No bootstrap required, opened a new log
I20260811 09:11:11.534601 22121 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5b3ea92d2ce64afdbce7d2182e00ca06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8d3967ffc3e4bda924776881f306353" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 44501 } } peers { permanent_uuid: "5b3ea92d2ce64afdbce7d2182e00ca06" member_type: VOTER last_known_addr { host: "127.28.154.189" port: 33105 } } peers { permanent_uuid: "802c96530cd3461b8f50680cc4b69eee" member_type: VOTER last_known_addr { host: "127.28.154.188" port: 38397 } }
I20260811 09:11:11.534955 22121 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5b3ea92d2ce64afdbce7d2182e00ca06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:11:11.535079 22121 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5b3ea92d2ce64afdbce7d2182e00ca06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b3ea92d2ce64afdbce7d2182e00ca06, State: Initialized, Role: FOLLOWER
I20260811 09:11:11.535696 22121 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5b3ea92d2ce64afdbce7d2182e00ca06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8d3967ffc3e4bda924776881f306353" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 44501 } } peers { permanent_uuid: "5b3ea92d2ce64afdbce7d2182e00ca06" member_type: VOTER last_known_addr { host: "127.28.154.189" port: 33105 } } peers { permanent_uuid: "802c96530cd3461b8f50680cc4b69eee" member_type: VOTER last_known_addr { host: "127.28.154.188" port: 38397 } }
I20260811 09:11:11.536639 22330 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b3ea92d2ce64afdbce7d2182e00ca06 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8d3967ffc3e4bda924776881f306353" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 44501 } } peers { permanent_uuid: "5b3ea92d2ce64afdbce7d2182e00ca06" member_type: VOTER last_known_addr { host: "127.28.154.189" port: 33105 } } peers { permanent_uuid: "802c96530cd3461b8f50680cc4b69eee" member_type: VOTER last_known_addr { host: "127.28.154.188" port: 38397 } } }
I20260811 09:11:11.537065 22330 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b3ea92d2ce64afdbce7d2182e00ca06 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 09:11:11.538152 22121 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5b3ea92d2ce64afdbce7d2182e00ca06 [sys.catalog]: configured and running, proceeding with master startup.
W20260811 09:11:11.553915 22185 tablet.cc:2369] T 00000000000000000000000000000000 P 802c96530cd3461b8f50680cc4b69eee: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 09:11:11.556736 22000 master_runner.cc:263] No leader master found from master 5b3ea92d2ce64afdbce7d2182e00ca06
I20260811 09:11:11.556887 22000 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260811 09:11:11.562845 22347 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5b3ea92d2ce64afdbce7d2182e00ca06: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 09:11:11.563120 22347 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260811 09:11:11.739696 22036 tablet.cc:2369] T 00000000000000000000000000000000 P 5b3ea92d2ce64afdbce7d2182e00ca06: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 09:11:11.759588 22290 raft_consensus.cc:482] T 00000000000000000000000000000000 P 802c96530cd3461b8f50680cc4b69eee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:11:11.759945 22290 raft_consensus.cc:504] T 00000000000000000000000000000000 P 802c96530cd3461b8f50680cc4b69eee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8d3967ffc3e4bda924776881f306353" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 44501 } } peers { permanent_uuid: "5b3ea92d2ce64afdbce7d2182e00ca06" member_type: VOTER last_known_addr { host: "127.28.154.189" port: 33105 } } peers { permanent_uuid: "802c96530cd3461b8f50680cc4b69eee" member_type: VOTER last_known_addr { host: "127.28.154.188" port: 38397 } }
I20260811 09:11:11.762068 22290 leader_election.cc:288] T 00000000000000000000000000000000 P 802c96530cd3461b8f50680cc4b69eee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8d3967ffc3e4bda924776881f306353 (127.28.154.190:44501), 5b3ea92d2ce64afdbce7d2182e00ca06 (127.28.154.189:33105)
I20260811 09:11:11.763320 21929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "802c96530cd3461b8f50680cc4b69eee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8d3967ffc3e4bda924776881f306353" is_pre_election: true
I20260811 09:11:11.764125 21929 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e8d3967ffc3e4bda924776881f306353 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 802c96530cd3461b8f50680cc4b69eee in term 0.
I20260811 09:11:11.765192 22180 leader_election.cc:302] T 00000000000000000000000000000000 P 802c96530cd3461b8f50680cc4b69eee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 802c96530cd3461b8f50680cc4b69eee, e8d3967ffc3e4bda924776881f306353; no voters:
I20260811 09:11:11.765803 22290 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 802c96530cd3461b8f50680cc4b69eee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 09:11:11.765972 22290 raft_consensus.cc:482] T 00000000000000000000000000000000 P 802c96530cd3461b8f50680cc4b69eee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 09:11:11.766077 22290 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 802c96530cd3461b8f50680cc4b69eee [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:11:11.769557 22075 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "802c96530cd3461b8f50680cc4b69eee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b3ea92d2ce64afdbce7d2182e00ca06" is_pre_election: true
I20260811 09:11:11.770239 22075 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5b3ea92d2ce64afdbce7d2182e00ca06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 802c96530cd3461b8f50680cc4b69eee in term 0.
I20260811 09:11:11.773993 22290 raft_consensus.cc:504] T 00000000000000000000000000000000 P 802c96530cd3461b8f50680cc4b69eee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8d3967ffc3e4bda924776881f306353" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 44501 } } peers { permanent_uuid: "5b3ea92d2ce64afdbce7d2182e00ca06" member_type: VOTER last_known_addr { host: "127.28.154.189" port: 33105 } } peers { permanent_uuid: "802c96530cd3461b8f50680cc4b69eee" member_type: VOTER last_known_addr { host: "127.28.154.188" port: 38397 } }
I20260811 09:11:11.775389 21929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "802c96530cd3461b8f50680cc4b69eee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8d3967ffc3e4bda924776881f306353"
I20260811 09:11:11.775635 21929 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e8d3967ffc3e4bda924776881f306353 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:11:11.779191 21929 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e8d3967ffc3e4bda924776881f306353 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 802c96530cd3461b8f50680cc4b69eee in term 1.
I20260811 09:11:11.779868 22180 leader_election.cc:302] T 00000000000000000000000000000000 P 802c96530cd3461b8f50680cc4b69eee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 802c96530cd3461b8f50680cc4b69eee, e8d3967ffc3e4bda924776881f306353; no voters:
I20260811 09:11:11.785619 22075 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "802c96530cd3461b8f50680cc4b69eee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5b3ea92d2ce64afdbce7d2182e00ca06"
I20260811 09:11:11.785861 22075 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5b3ea92d2ce64afdbce7d2182e00ca06 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:11:11.789590 22075 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5b3ea92d2ce64afdbce7d2182e00ca06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 802c96530cd3461b8f50680cc4b69eee in term 1.
I20260811 09:11:11.790627 22290 leader_election.cc:288] T 00000000000000000000000000000000 P 802c96530cd3461b8f50680cc4b69eee [CANDIDATE]: Term 1 election: Requested vote from peers e8d3967ffc3e4bda924776881f306353 (127.28.154.190:44501), 5b3ea92d2ce64afdbce7d2182e00ca06 (127.28.154.189:33105)
I20260811 09:11:11.791042 22290 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 802c96530cd3461b8f50680cc4b69eee [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:11:11.791774 22290 raft_consensus.cc:686] T 00000000000000000000000000000000 P 802c96530cd3461b8f50680cc4b69eee [term 1 LEADER]: Becoming Leader. State: Replica: 802c96530cd3461b8f50680cc4b69eee, State: Running, Role: LEADER
I20260811 09:11:11.796897 22290 consensus_queue.cc:237] T 00000000000000000000000000000000 P 802c96530cd3461b8f50680cc4b69eee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8d3967ffc3e4bda924776881f306353" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 44501 } } peers { permanent_uuid: "5b3ea92d2ce64afdbce7d2182e00ca06" member_type: VOTER last_known_addr { host: "127.28.154.189" port: 33105 } } peers { permanent_uuid: "802c96530cd3461b8f50680cc4b69eee" member_type: VOTER last_known_addr { host: "127.28.154.188" port: 38397 } }
I20260811 09:11:11.820694 22349 sys_catalog.cc:455] T 00000000000000000000000000000000 P 802c96530cd3461b8f50680cc4b69eee [sys.catalog]: SysCatalogTable state changed. Reason: New leader 802c96530cd3461b8f50680cc4b69eee. Latest consensus state: current_term: 1 leader_uuid: "802c96530cd3461b8f50680cc4b69eee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8d3967ffc3e4bda924776881f306353" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 44501 } } peers { permanent_uuid: "5b3ea92d2ce64afdbce7d2182e00ca06" member_type: VOTER last_known_addr { host: "127.28.154.189" port: 33105 } } peers { permanent_uuid: "802c96530cd3461b8f50680cc4b69eee" member_type: VOTER last_known_addr { host: "127.28.154.188" port: 38397 } } }
I20260811 09:11:11.821285 22349 sys_catalog.cc:458] T 00000000000000000000000000000000 P 802c96530cd3461b8f50680cc4b69eee [sys.catalog]: This master's current role is: LEADER
I20260811 09:11:11.837087 22361 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:11:11.839419 22361 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:11:11.855799 22075 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5b3ea92d2ce64afdbce7d2182e00ca06 [term 1 FOLLOWER]: Refusing update from remote peer 802c96530cd3461b8f50680cc4b69eee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:11:11.856946 22349 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 802c96530cd3461b8f50680cc4b69eee [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b3ea92d2ce64afdbce7d2182e00ca06" member_type: VOTER last_known_addr { host: "127.28.154.189" port: 33105 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:11:11.864413 21929 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e8d3967ffc3e4bda924776881f306353 [term 1 FOLLOWER]: Refusing update from remote peer 802c96530cd3461b8f50680cc4b69eee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:11:11.865422 22349 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 802c96530cd3461b8f50680cc4b69eee [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8d3967ffc3e4bda924776881f306353" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 44501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:11:11.883751 22282 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8d3967ffc3e4bda924776881f306353 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 802c96530cd3461b8f50680cc4b69eee. Latest consensus state: current_term: 1 leader_uuid: "802c96530cd3461b8f50680cc4b69eee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8d3967ffc3e4bda924776881f306353" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 44501 } } peers { permanent_uuid: "5b3ea92d2ce64afdbce7d2182e00ca06" member_type: VOTER last_known_addr { host: "127.28.154.189" port: 33105 } } peers { permanent_uuid: "802c96530cd3461b8f50680cc4b69eee" member_type: VOTER last_known_addr { host: "127.28.154.188" port: 38397 } } }
I20260811 09:11:11.884104 22282 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8d3967ffc3e4bda924776881f306353 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 09:11:11.896232 22362 mvcc.cc:205] Tried to move back new op lower bound from 7317256076709330944 to 7317256076511924224. Current Snapshot: MvccSnapshot[applied={T|T < 7317256076709330944}]
I20260811 09:11:11.902904 22330 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b3ea92d2ce64afdbce7d2182e00ca06 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 802c96530cd3461b8f50680cc4b69eee. Latest consensus state: current_term: 1 leader_uuid: "802c96530cd3461b8f50680cc4b69eee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8d3967ffc3e4bda924776881f306353" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 44501 } } peers { permanent_uuid: "5b3ea92d2ce64afdbce7d2182e00ca06" member_type: VOTER last_known_addr { host: "127.28.154.189" port: 33105 } } peers { permanent_uuid: "802c96530cd3461b8f50680cc4b69eee" member_type: VOTER last_known_addr { host: "127.28.154.188" port: 38397 } } }
I20260811 09:11:11.903277 22330 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b3ea92d2ce64afdbce7d2182e00ca06 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 09:11:11.908311 22282 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8d3967ffc3e4bda924776881f306353 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "802c96530cd3461b8f50680cc4b69eee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8d3967ffc3e4bda924776881f306353" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 44501 } } peers { permanent_uuid: "5b3ea92d2ce64afdbce7d2182e00ca06" member_type: VOTER last_known_addr { host: "127.28.154.189" port: 33105 } } peers { permanent_uuid: "802c96530cd3461b8f50680cc4b69eee" member_type: VOTER last_known_addr { host: "127.28.154.188" port: 38397 } } }
I20260811 09:11:11.908649 22282 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8d3967ffc3e4bda924776881f306353 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 09:11:11.913399 22361 catalog_manager.cc:1347] Generated new cluster ID: 7815e8283d8243e2ae4a09379c927516
I20260811 09:11:11.913534 22361 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:11:11.913507 22349 sys_catalog.cc:455] T 00000000000000000000000000000000 P 802c96530cd3461b8f50680cc4b69eee [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "802c96530cd3461b8f50680cc4b69eee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8d3967ffc3e4bda924776881f306353" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 44501 } } peers { permanent_uuid: "5b3ea92d2ce64afdbce7d2182e00ca06" member_type: VOTER last_known_addr { host: "127.28.154.189" port: 33105 } } peers { permanent_uuid: "802c96530cd3461b8f50680cc4b69eee" member_type: VOTER last_known_addr { host: "127.28.154.188" port: 38397 } } }
I20260811 09:11:11.913784 22349 sys_catalog.cc:458] T 00000000000000000000000000000000 P 802c96530cd3461b8f50680cc4b69eee [sys.catalog]: This master's current role is: LEADER
I20260811 09:11:11.914119 22349 sys_catalog.cc:455] T 00000000000000000000000000000000 P 802c96530cd3461b8f50680cc4b69eee [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "802c96530cd3461b8f50680cc4b69eee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8d3967ffc3e4bda924776881f306353" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 44501 } } peers { permanent_uuid: "5b3ea92d2ce64afdbce7d2182e00ca06" member_type: VOTER last_known_addr { host: "127.28.154.189" port: 33105 } } peers { permanent_uuid: "802c96530cd3461b8f50680cc4b69eee" member_type: VOTER last_known_addr { host: "127.28.154.188" port: 38397 } } }
I20260811 09:11:11.914348 22349 sys_catalog.cc:458] T 00000000000000000000000000000000 P 802c96530cd3461b8f50680cc4b69eee [sys.catalog]: This master's current role is: LEADER
I20260811 09:11:11.916620 22363 mvcc.cc:205] Tried to move back new op lower bound from 7317256076709330944 to 7317256076511924224. Current Snapshot: MvccSnapshot[applied={T|T < 7317256076709330944}]
I20260811 09:11:11.921484 22330 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b3ea92d2ce64afdbce7d2182e00ca06 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "802c96530cd3461b8f50680cc4b69eee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8d3967ffc3e4bda924776881f306353" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 44501 } } peers { permanent_uuid: "5b3ea92d2ce64afdbce7d2182e00ca06" member_type: VOTER last_known_addr { host: "127.28.154.189" port: 33105 } } peers { permanent_uuid: "802c96530cd3461b8f50680cc4b69eee" member_type: VOTER last_known_addr { host: "127.28.154.188" port: 38397 } } }
I20260811 09:11:11.921813 22330 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b3ea92d2ce64afdbce7d2182e00ca06 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 09:11:11.937996 22361 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:11:11.939646 22361 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:11:11.960625 22361 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 802c96530cd3461b8f50680cc4b69eee: Generated new TSK 0
I20260811 09:11:11.962307 22361 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 09:11:12.485599 22311 catalog_manager.cc:1259] Loaded cluster ID: 7815e8283d8243e2ae4a09379c927516
I20260811 09:11:12.485718 22311 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e8d3967ffc3e4bda924776881f306353: loading cluster ID for follower catalog manager: success
I20260811 09:11:12.488245 22311 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e8d3967ffc3e4bda924776881f306353: acquiring CA information for follower catalog manager: success
I20260811 09:11:12.490159 22311 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e8d3967ffc3e4bda924776881f306353: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 09:11:12.565351 22347 catalog_manager.cc:1259] Loaded cluster ID: 7815e8283d8243e2ae4a09379c927516
I20260811 09:11:12.565470 22347 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 5b3ea92d2ce64afdbce7d2182e00ca06: loading cluster ID for follower catalog manager: success
I20260811 09:11:12.568018 22347 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 5b3ea92d2ce64afdbce7d2182e00ca06: acquiring CA information for follower catalog manager: success
I20260811 09:11:12.569903 22347 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 5b3ea92d2ce64afdbce7d2182e00ca06: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 09:11:12.594684 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 21859
I20260811 09:11:12.596570 21966 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:11:12.707058 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 21859
I20260811 09:11:12.739065 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 22000
I20260811 09:11:12.740231 22113 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:11:12.854758 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 22000
W20260811 09:11:12.881304 22178 connection.cc:535] client connection to 127.28.154.189:33105 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 09:11:12.881528 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 22137
W20260811 09:11:12.881587 22178 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 09:11:12.883188 22260 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:11:12.992851 22400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:12.882728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.189:33105 (local address 127.0.0.1:60128)
0811 09:11:12.883149 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:12.992564 (+109415us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.189:33105: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":196,"threads_started":1}
W20260811 09:11:12.993110 22178 consensus_peers.cc:487] T 00000000000000000000000000000000 P 802c96530cd3461b8f50680cc4b69eee -> Peer 5b3ea92d2ce64afdbce7d2182e00ca06 (127.28.154.189:33105): Couldn't send request to peer 5b3ea92d2ce64afdbce7d2182e00ca06. Status: Network error: Client connection negotiation failed: client connection to 127.28.154.189:33105: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 09:11:12.993228 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 22137
2026-08-11T09:11:13Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (4823 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786439250192320-29290-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:11:13 dist-test-slave-xt4k krb5kdc[22407](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:11:13 dist-test-slave-xt4k krb5kdc[22407](info): set up 2 sockets
Aug 11 09:11:13 dist-test-slave-xt4k krb5kdc[22407](info): commencing operation
krb5kdc: starting...
W20260811 09:11:14.606277 29290 mini_kdc.cc:121] Time spent starting KDC: real 1.564s user 0.000s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786439250192320-29290-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786439250192320-29290-0/krb5kdc/test-user.keytab.
Aug 11 09:11:14 dist-test-slave-xt4k krb5kdc[22407](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439474, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:11:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:11:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.154.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.154.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.154.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab.
Entry for principal kudu/127.28.154.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab.
I20260811 09:11:14.913055 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786439250192320-29290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786439250192320-29290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786439250192320-29290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.154.190:39541
--webserver_interface=127.28.154.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab
--principal=kudu/127.28.154.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:39171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.154.190:39541
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:11:15.277761 22427 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:11:15.278087 22427 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:11:15.278206 22427 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:11:15.285738 22427 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:11:15.285871 22427 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:11:15.285923 22427 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:11:15.286005 22427 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:11:15.286068 22427 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:11:15.296772 22427 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:39171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786439250192320-29290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786439250192320-29290-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.154.190:39541
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab
--principal=kudu/127.28.154.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.154.190:39541
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.154.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786439250192320-29290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:11:15.298209 22427 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:11:15.299605 22427 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:11:15.312880 22436 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:11:15.312912 22438 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:11:15.312816 22435 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:11:15.313723 22427 server_base.cc:1034] running on GCE node
I20260811 09:11:15.314632 22427 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:11:15.316430 22427 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:11:15.317679 22427 hybrid_clock.cc:648] HybridClock initialized: now 1786439475317618 us; error 50 us; skew 500 ppm
I20260811 09:11:15.318084 22427 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:11:15 dist-test-slave-xt4k krb5kdc[22407](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439475, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:11:15.321696 22427 init.cc:377] Logged in from keytab as kudu/127.28.154.190@KRBTEST.COM (short username kudu)
I20260811 09:11:15.323410 22427 webserver.cc:460] Webserver started at http://127.28.154.190:37477/ using document root <none> and password file <none>
I20260811 09:11:15.324172 22427 fs_manager.cc:359] Metadata directory not provided
I20260811 09:11:15.324301 22427 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:11:15.324651 22427 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:11:15.328214 22427 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786439250192320-29290-0/minicluster-data/master-0/data/instance:
uuid: "64ef2d04d4244a7ab68fa5250c2b34dc"
format_stamp: "Formatted at 2026-08-11 09:11:15 on dist-test-slave-xt4k"
server_key: "68d9e6673e580c8b71c4db179d96fc7a"
server_key_iv: "9d9d0594aa6226dc3de067cf688d3d3d"
server_key_version: "encryptionkey@0"
I20260811 09:11:15.329241 22427 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786439250192320-29290-0/minicluster-data/master-0/wal/instance:
uuid: "64ef2d04d4244a7ab68fa5250c2b34dc"
format_stamp: "Formatted at 2026-08-11 09:11:15 on dist-test-slave-xt4k"
server_key: "68d9e6673e580c8b71c4db179d96fc7a"
server_key_iv: "9d9d0594aa6226dc3de067cf688d3d3d"
server_key_version: "encryptionkey@0"
I20260811 09:11:15.334944 22427 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260811 09:11:15.339181 22450 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:11:15.340854 22427 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 09:11:15.341048 22427 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786439250192320-29290-0/minicluster-data/master-0/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786439250192320-29290-0/minicluster-data/master-0/wal
uuid: "64ef2d04d4244a7ab68fa5250c2b34dc"
format_stamp: "Formatted at 2026-08-11 09:11:15 on dist-test-slave-xt4k"
server_key: "68d9e6673e580c8b71c4db179d96fc7a"
server_key_iv: "9d9d0594aa6226dc3de067cf688d3d3d"
server_key_version: "encryptionkey@0"
I20260811 09:11:15.341260 22427 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786439250192320-29290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786439250192320-29290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786439250192320-29290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:11:15.365197 22427 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:11:15.370731 22427 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:11:15.371066 22427 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:11:15.393195 22427 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.190:39541
I20260811 09:11:15.393217 22536 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.190:39541 every 8 connection(s)
I20260811 09:11:15.394702 22427 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
I20260811 09:11:15.399673 22538 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:11:15.403662 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 22427
I20260811 09:11:15.403927 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786439250192320-29290-0/minicluster-data/master-0/wal/instance
I20260811 09:11:15.404840 29290 external_mini_cluster.cc:1428] Setting key 42f3cc4d147226a15beef13db7bcd650
I20260811 09:11:15.414419 22538 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64ef2d04d4244a7ab68fa5250c2b34dc: Bootstrap starting.
I20260811 09:11:15.420903 22538 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 64ef2d04d4244a7ab68fa5250c2b34dc: Neither blocks nor log segments found. Creating new log.
I20260811 09:11:15.422608 22538 log.cc:824] T 00000000000000000000000000000000 P 64ef2d04d4244a7ab68fa5250c2b34dc: Log is configured to *not* fsync() on all Append() calls
Aug 11 09:11:15 dist-test-slave-xt4k krb5kdc[22407](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439474, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.154.190@KRBTEST.COM
I20260811 09:11:15.426667 22538 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64ef2d04d4244a7ab68fa5250c2b34dc: No bootstrap required, opened a new log
I20260811 09:11:15.436440 22541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:15.407034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49936 (local address 127.28.154.190:39541)
0811 09:11:15.407877 (+ 843us) server_negotiation.cc:206] Beginning negotiation
0811 09:11:15.407889 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 09:11:15.407934 (+ 45us) server_negotiation.cc:407] Connection header received
0811 09:11:15.409867 (+ 1933us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:11:15.409897 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:11:15.410379 (+ 482us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:11:15.410778 (+ 399us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:11:15.412968 (+ 2190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:15.415223 (+ 2255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:15.418646 (+ 3423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:15.419461 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:15.419796 (+ 335us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:15.426243 (+ 6447us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:11:15.426257 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:11:15.426278 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:11:15.426333 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:11:15.430876 (+ 4543us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:11:15.431883 (+ 1007us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:11:15.431893 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:11:15.431903 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:11:15.432178 (+ 275us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:11:15.432685 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:11:15.432691 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:11:15.432695 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:11:15.433542 (+ 847us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:11:15.433867 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0811 09:11:15.434335 (+ 468us) server_negotiation.cc:299] Negotiation successful
0811 09:11:15.434847 (+ 512us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":599,"thread_start_us":372,"threads_started":1}
I20260811 09:11:15.437786 22538 raft_consensus.cc:348] T 00000000000000000000000000000000 P 64ef2d04d4244a7ab68fa5250c2b34dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64ef2d04d4244a7ab68fa5250c2b34dc" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 39541 } }
I20260811 09:11:15.438294 22538 raft_consensus.cc:374] T 00000000000000000000000000000000 P 64ef2d04d4244a7ab68fa5250c2b34dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:11:15.438434 22538 raft_consensus.cc:729] T 00000000000000000000000000000000 P 64ef2d04d4244a7ab68fa5250c2b34dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64ef2d04d4244a7ab68fa5250c2b34dc, State: Initialized, Role: FOLLOWER
I20260811 09:11:15.439034 22538 consensus_queue.cc:260] T 00000000000000000000000000000000 P 64ef2d04d4244a7ab68fa5250c2b34dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64ef2d04d4244a7ab68fa5250c2b34dc" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 39541 } }
I20260811 09:11:15.439332 22538 raft_consensus.cc:388] T 00000000000000000000000000000000 P 64ef2d04d4244a7ab68fa5250c2b34dc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:11:15.439527 22538 raft_consensus.cc:482] T 00000000000000000000000000000000 P 64ef2d04d4244a7ab68fa5250c2b34dc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:11:15.439736 22538 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 64ef2d04d4244a7ab68fa5250c2b34dc [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:11:15.441228 29290 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 09:11:15.442313 22538 raft_consensus.cc:504] T 00000000000000000000000000000000 P 64ef2d04d4244a7ab68fa5250c2b34dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64ef2d04d4244a7ab68fa5250c2b34dc" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 39541 } }
I20260811 09:11:15.442873 22538 leader_election.cc:302] T 00000000000000000000000000000000 P 64ef2d04d4244a7ab68fa5250c2b34dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 64ef2d04d4244a7ab68fa5250c2b34dc; no voters:
I20260811 09:11:15.443486 22538 leader_election.cc:288] T 00000000000000000000000000000000 P 64ef2d04d4244a7ab68fa5250c2b34dc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:11:15.443637 22544 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 64ef2d04d4244a7ab68fa5250c2b34dc [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:11:15.445339 22544 raft_consensus.cc:686] T 00000000000000000000000000000000 P 64ef2d04d4244a7ab68fa5250c2b34dc [term 1 LEADER]: Becoming Leader. State: Replica: 64ef2d04d4244a7ab68fa5250c2b34dc, State: Running, Role: LEADER
I20260811 09:11:15.446053 22544 consensus_queue.cc:237] T 00000000000000000000000000000000 P 64ef2d04d4244a7ab68fa5250c2b34dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64ef2d04d4244a7ab68fa5250c2b34dc" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 39541 } }
I20260811 09:11:15.446508 22538 sys_catalog.cc:564] T 00000000000000000000000000000000 P 64ef2d04d4244a7ab68fa5250c2b34dc [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:11:15.449261 22546 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64ef2d04d4244a7ab68fa5250c2b34dc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "64ef2d04d4244a7ab68fa5250c2b34dc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64ef2d04d4244a7ab68fa5250c2b34dc" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 39541 } } }
I20260811 09:11:15.449172 22547 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64ef2d04d4244a7ab68fa5250c2b34dc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 64ef2d04d4244a7ab68fa5250c2b34dc. Latest consensus state: current_term: 1 leader_uuid: "64ef2d04d4244a7ab68fa5250c2b34dc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64ef2d04d4244a7ab68fa5250c2b34dc" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 39541 } } }
I20260811 09:11:15.449709 22547 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64ef2d04d4244a7ab68fa5250c2b34dc [sys.catalog]: This master's current role is: LEADER
I20260811 09:11:15.449709 22546 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64ef2d04d4244a7ab68fa5250c2b34dc [sys.catalog]: This master's current role is: LEADER
I20260811 09:11:15.450624 22562 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:11:15.456456 22562 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:11:15.466156 22562 catalog_manager.cc:1347] Generated new cluster ID: 8895a4b741294f95b5e7a0d40af1af6c
I20260811 09:11:15.466341 22562 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:11:15.481806 22562 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:11:15.483253 22562 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:11:15.492700 22562 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 64ef2d04d4244a7ab68fa5250c2b34dc: Generated new TSK 0
I20260811 09:11:15.493887 22562 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 09:11:15.505815 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 22427
I20260811 09:11:15.507349 22532 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:11:15.601823 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 22427
2026-08-11T09:11:15Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (2609 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:11:15 dist-test-slave-xt4k krb5kdc[22586](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:11:15 dist-test-slave-xt4k krb5kdc[22586](info): set up 2 sockets
Aug 11 09:11:15 dist-test-slave-xt4k krb5kdc[22586](info): commencing operation
krb5kdc: starting...
W20260811 09:11:16.928678 29290 mini_kdc.cc:121] Time spent starting KDC: real 1.277s user 0.006s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/krb5kdc/test-user.keytab.
Aug 11 09:11:17 dist-test-slave-xt4k krb5kdc[22586](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439477, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:11:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:11:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.154.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.154.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.154.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab.
Entry for principal kudu/127.28.154.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab.
I20260811 09:11:17.225735 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.154.190:36117
--webserver_interface=127.28.154.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab
--principal=kudu/127.28.154.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:32851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.154.190:36117
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:11:17.575518 22610 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:11:17.576205 22610 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:11:17.576400 22610 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:11:17.584034 22610 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:11:17.584192 22610 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:11:17.584275 22610 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:11:17.584389 22610 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:11:17.584458 22610 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:11:17.595562 22610 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:32851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.154.190:36117
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab
--principal=kudu/127.28.154.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.154.190:36117
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.154.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:11:17.597038 22610 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:11:17.598353 22610 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:11:17.610316 22618 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:11:17.610332 22619 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:11:17.610342 22621 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:11:17.611183 22610 server_base.cc:1034] running on GCE node
I20260811 09:11:17.611943 22610 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:11:17.613516 22610 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:11:17.614753 22610 hybrid_clock.cc:648] HybridClock initialized: now 1786439477614698 us; error 36 us; skew 500 ppm
I20260811 09:11:17.615124 22610 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:11:17 dist-test-slave-xt4k krb5kdc[22586](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439477, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:11:17.621675 22610 init.cc:377] Logged in from keytab as kudu/127.28.154.190@KRBTEST.COM (short username kudu)
I20260811 09:11:17.623426 22610 webserver.cc:460] Webserver started at http://127.28.154.190:37189/ using document root <none> and password file <none>
I20260811 09:11:17.624173 22610 fs_manager.cc:359] Metadata directory not provided
I20260811 09:11:17.624289 22610 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:11:17.624646 22610 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:11:17.627851 22610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/master-0/data/instance:
uuid: "adc54f73b9dc4ef09d9ede64155cd1e5"
format_stamp: "Formatted at 2026-08-11 09:11:17 on dist-test-slave-xt4k"
server_key: "546556857f41fa52df510ea4579ae8bf"
server_key_iv: "56d120bb3d2faf25c7812b93fdb857ed"
server_key_version: "encryptionkey@0"
I20260811 09:11:17.628688 22610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/master-0/wal/instance:
uuid: "adc54f73b9dc4ef09d9ede64155cd1e5"
format_stamp: "Formatted at 2026-08-11 09:11:17 on dist-test-slave-xt4k"
server_key: "546556857f41fa52df510ea4579ae8bf"
server_key_iv: "56d120bb3d2faf25c7812b93fdb857ed"
server_key_version: "encryptionkey@0"
I20260811 09:11:17.634400 22610 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 09:11:17.638388 22635 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:11:17.639896 22610 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 09:11:17.640091 22610 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/master-0/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/master-0/wal
uuid: "adc54f73b9dc4ef09d9ede64155cd1e5"
format_stamp: "Formatted at 2026-08-11 09:11:17 on dist-test-slave-xt4k"
server_key: "546556857f41fa52df510ea4579ae8bf"
server_key_iv: "56d120bb3d2faf25c7812b93fdb857ed"
server_key_version: "encryptionkey@0"
I20260811 09:11:17.640297 22610 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:11:17.662173 22610 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:11:17.666713 22610 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:11:17.667007 22610 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:11:17.687439 22610 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.190:36117
I20260811 09:11:17.687487 22721 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.190:36117 every 8 connection(s)
I20260811 09:11:17.689002 22610 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
I20260811 09:11:17.693064 22722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:11:17.695320 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 22610
I20260811 09:11:17.695540 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/master-0/wal/instance
I20260811 09:11:17.696290 29290 external_mini_cluster.cc:1428] Setting key 7e4f7caf556bd078f57b248e7db0c295
I20260811 09:11:17.707453 22722 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P adc54f73b9dc4ef09d9ede64155cd1e5: Bootstrap starting.
I20260811 09:11:17.711540 22722 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P adc54f73b9dc4ef09d9ede64155cd1e5: Neither blocks nor log segments found. Creating new log.
I20260811 09:11:17.712690 22722 log.cc:824] T 00000000000000000000000000000000 P adc54f73b9dc4ef09d9ede64155cd1e5: Log is configured to *not* fsync() on all Append() calls
Aug 11 09:11:17 dist-test-slave-xt4k krb5kdc[22586](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439477, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.154.190@KRBTEST.COM
I20260811 09:11:17.717907 22722 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P adc54f73b9dc4ef09d9ede64155cd1e5: No bootstrap required, opened a new log
I20260811 09:11:17.727217 22728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:17.699282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58686 (local address 127.28.154.190:36117)
0811 09:11:17.700155 (+ 873us) server_negotiation.cc:206] Beginning negotiation
0811 09:11:17.700174 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 09:11:17.700227 (+ 53us) server_negotiation.cc:407] Connection header received
0811 09:11:17.702920 (+ 2693us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:11:17.702955 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:11:17.703573 (+ 618us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:11:17.704089 (+ 516us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:11:17.705599 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:17.707646 (+ 2047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:17.709325 (+ 1679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:17.710408 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:17.710629 (+ 221us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:17.715443 (+ 4814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:11:17.715459 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:11:17.715490 (+ 31us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:11:17.715568 (+ 78us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:11:17.720594 (+ 5026us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:11:17.721632 (+ 1038us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:11:17.721648 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:11:17.721659 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:11:17.722167 (+ 508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:11:17.722798 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:11:17.722804 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:11:17.722807 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:11:17.723655 (+ 848us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:11:17.724032 (+ 377us) server_negotiation.cc:1035] Waiting for connection context
0811 09:11:17.724565 (+ 533us) server_negotiation.cc:299] Negotiation successful
0811 09:11:17.725100 (+ 535us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":608,"thread_start_us":374,"threads_started":1}
I20260811 09:11:17.730036 22722 raft_consensus.cc:348] T 00000000000000000000000000000000 P adc54f73b9dc4ef09d9ede64155cd1e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adc54f73b9dc4ef09d9ede64155cd1e5" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 36117 } }
I20260811 09:11:17.730569 22722 raft_consensus.cc:374] T 00000000000000000000000000000000 P adc54f73b9dc4ef09d9ede64155cd1e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:11:17.730708 22722 raft_consensus.cc:729] T 00000000000000000000000000000000 P adc54f73b9dc4ef09d9ede64155cd1e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: adc54f73b9dc4ef09d9ede64155cd1e5, State: Initialized, Role: FOLLOWER
I20260811 09:11:17.731287 22722 consensus_queue.cc:260] T 00000000000000000000000000000000 P adc54f73b9dc4ef09d9ede64155cd1e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adc54f73b9dc4ef09d9ede64155cd1e5" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 36117 } }
I20260811 09:11:17.731516 22722 raft_consensus.cc:388] T 00000000000000000000000000000000 P adc54f73b9dc4ef09d9ede64155cd1e5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:11:17.731626 22722 raft_consensus.cc:482] T 00000000000000000000000000000000 P adc54f73b9dc4ef09d9ede64155cd1e5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:11:17.731755 22722 raft_consensus.cc:3037] T 00000000000000000000000000000000 P adc54f73b9dc4ef09d9ede64155cd1e5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:11:17.734900 22722 raft_consensus.cc:504] T 00000000000000000000000000000000 P adc54f73b9dc4ef09d9ede64155cd1e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adc54f73b9dc4ef09d9ede64155cd1e5" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 36117 } }
I20260811 09:11:17.735533 22722 leader_election.cc:302] T 00000000000000000000000000000000 P adc54f73b9dc4ef09d9ede64155cd1e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: adc54f73b9dc4ef09d9ede64155cd1e5; no voters:
I20260811 09:11:17.736275 22722 leader_election.cc:288] T 00000000000000000000000000000000 P adc54f73b9dc4ef09d9ede64155cd1e5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:11:17.736388 22730 raft_consensus.cc:2781] T 00000000000000000000000000000000 P adc54f73b9dc4ef09d9ede64155cd1e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:11:17.737507 22730 raft_consensus.cc:686] T 00000000000000000000000000000000 P adc54f73b9dc4ef09d9ede64155cd1e5 [term 1 LEADER]: Becoming Leader. State: Replica: adc54f73b9dc4ef09d9ede64155cd1e5, State: Running, Role: LEADER
I20260811 09:11:17.738090 22730 consensus_queue.cc:237] T 00000000000000000000000000000000 P adc54f73b9dc4ef09d9ede64155cd1e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adc54f73b9dc4ef09d9ede64155cd1e5" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 36117 } }
I20260811 09:11:17.738957 22722 sys_catalog.cc:564] T 00000000000000000000000000000000 P adc54f73b9dc4ef09d9ede64155cd1e5 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:11:17.741111 22733 sys_catalog.cc:455] T 00000000000000000000000000000000 P adc54f73b9dc4ef09d9ede64155cd1e5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "adc54f73b9dc4ef09d9ede64155cd1e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adc54f73b9dc4ef09d9ede64155cd1e5" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 36117 } } }
I20260811 09:11:17.741521 22733 sys_catalog.cc:458] T 00000000000000000000000000000000 P adc54f73b9dc4ef09d9ede64155cd1e5 [sys.catalog]: This master's current role is: LEADER
I20260811 09:11:17.742445 22737 sys_catalog.cc:455] T 00000000000000000000000000000000 P adc54f73b9dc4ef09d9ede64155cd1e5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader adc54f73b9dc4ef09d9ede64155cd1e5. Latest consensus state: current_term: 1 leader_uuid: "adc54f73b9dc4ef09d9ede64155cd1e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adc54f73b9dc4ef09d9ede64155cd1e5" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 36117 } } }
I20260811 09:11:17.742816 22737 sys_catalog.cc:458] T 00000000000000000000000000000000 P adc54f73b9dc4ef09d9ede64155cd1e5 [sys.catalog]: This master's current role is: LEADER
I20260811 09:11:17.745618 22742 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:11:17.753710 22742 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:11:17.763339 22742 catalog_manager.cc:1347] Generated new cluster ID: f632f10c6dc34ca093d24d59728d1a6b
I20260811 09:11:17.763474 22742 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:11:17.790392 22742 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:11:17.791851 22742 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:11:17.805243 22742 catalog_manager.cc:5868] T 00000000000000000000000000000000 P adc54f73b9dc4ef09d9ede64155cd1e5: Generated new TSK 0
I20260811 09:11:17.806102 22742 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.28.154.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.154.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.154.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.129.keytab.
Entry for principal kudu/127.28.154.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.129.keytab.
I20260811 09:11:17.888906 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.154.129:0
--local_ip_for_outbound_sockets=127.28.154.129
--webserver_interface=127.28.154.129
--webserver_port=0
--tserver_master_addrs=127.28.154.190:36117
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.129.keytab
--principal=kudu/127.28.154.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:32851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:11:18.253084 22767 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:11:18.253393 22767 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:11:18.253528 22767 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:11:18.261929 22767 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:11:18.262073 22767 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:11:18.262229 22767 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.154.129
I20260811 09:11:18.272459 22767 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:32851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.129.keytab
--principal=kudu/127.28.154.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.154.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.154.129
--webserver_port=0
--tserver_master_addrs=127.28.154.190:36117
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.154.129
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:11:18.273901 22767 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:11:18.275344 22767 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:11:18.289220 22778 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:11:18.289183 22781 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:11:18.289242 22783 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:11:18.290350 22767 server_base.cc:1034] running on GCE node
I20260811 09:11:18.291029 22767 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:11:18.292059 22767 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:11:18.293298 22767 hybrid_clock.cc:648] HybridClock initialized: now 1786439478293205 us; error 79 us; skew 500 ppm
I20260811 09:11:18.293709 22767 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:11:18 dist-test-slave-xt4k krb5kdc[22586](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439478, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:11:18.297255 22767 init.cc:377] Logged in from keytab as kudu/127.28.154.129@KRBTEST.COM (short username kudu)
I20260811 09:11:18.298888 22767 webserver.cc:460] Webserver started at http://127.28.154.129:40307/ using document root <none> and password file <none>
I20260811 09:11:18.299630 22767 fs_manager.cc:359] Metadata directory not provided
I20260811 09:11:18.299759 22767 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:11:18.300083 22767 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:11:18.303270 22767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-0/data/instance:
uuid: "e04ee1cf144a4460abe550ff84ff1b29"
format_stamp: "Formatted at 2026-08-11 09:11:18 on dist-test-slave-xt4k"
server_key: "c43fce2f65395065e27c92a5d3d25262"
server_key_iv: "5b5b0ddc94a7f4ad14f8de25489600d0"
server_key_version: "encryptionkey@0"
I20260811 09:11:18.304100 22767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-0/wal/instance:
uuid: "e04ee1cf144a4460abe550ff84ff1b29"
format_stamp: "Formatted at 2026-08-11 09:11:18 on dist-test-slave-xt4k"
server_key: "c43fce2f65395065e27c92a5d3d25262"
server_key_iv: "5b5b0ddc94a7f4ad14f8de25489600d0"
server_key_version: "encryptionkey@0"
I20260811 09:11:18.310076 22767 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.000s sys 0.004s
I20260811 09:11:18.313833 22794 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:11:18.315328 22767 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 09:11:18.315506 22767 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-0/wal
uuid: "e04ee1cf144a4460abe550ff84ff1b29"
format_stamp: "Formatted at 2026-08-11 09:11:18 on dist-test-slave-xt4k"
server_key: "c43fce2f65395065e27c92a5d3d25262"
server_key_iv: "5b5b0ddc94a7f4ad14f8de25489600d0"
server_key_version: "encryptionkey@0"
I20260811 09:11:18.315830 22767 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:11:18.347822 22767 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:11:18.352389 22767 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:11:18.352694 22767 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:11:18.353763 22767 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:11:18.355162 22767 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:11:18.355247 22767 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:11:18.355373 22767 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:11:18.355438 22767 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:11:18.387423 22767 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.129:44993
I20260811 09:11:18.387461 23005 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.129:44993 every 8 connection(s)
I20260811 09:11:18.388770 22767 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-0/data/info.pb
I20260811 09:11:18.391101 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 22767
I20260811 09:11:18.391350 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-0/wal/instance
I20260811 09:11:18.392189 29290 external_mini_cluster.cc:1428] Setting key ee15e4054f137a4fc856b88ff9f87848
Aug 11 09:11:18 dist-test-slave-xt4k krb5kdc[22586](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439478, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.129@KRBTEST.COM for kudu/127.28.154.190@KRBTEST.COM
I20260811 09:11:18.409467 23011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:18.391464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.129:43367 (local address 127.28.154.190:36117)
0811 09:11:18.391894 (+ 430us) server_negotiation.cc:206] Beginning negotiation
0811 09:11:18.391899 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:11:18.393207 (+ 1308us) server_negotiation.cc:407] Connection header received
0811 09:11:18.395268 (+ 2061us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:11:18.395274 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:11:18.395357 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:11:18.395449 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:11:18.397019 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:18.397732 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:18.399301 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:18.399696 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:18.399808 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:18.403379 (+ 3571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:11:18.403384 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:11:18.403387 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:11:18.403420 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:11:18.405953 (+ 2533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:11:18.406854 (+ 901us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:11:18.406859 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:11:18.406862 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:11:18.407005 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:11:18.407529 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:11:18.407533 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:11:18.407536 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:11:18.407871 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:11:18.408042 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0811 09:11:18.408857 (+ 815us) server_negotiation.cc:299] Negotiation successful
0811 09:11:18.409105 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":209,"threads_started":1}
I20260811 09:11:18.410336 23012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:18.392141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:36117 (local address 127.28.154.129:43367)
0811 09:11:18.392968 (+ 827us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:18.393034 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0811 09:11:18.394997 (+ 1963us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:11:18.395709 (+ 712us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:11:18.395718 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:11:18.396223 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:11:18.396842 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:11:18.396860 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:11:18.397900 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:11:18.397904 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:11:18.399145 (+ 1241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:11:18.399158 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:11:18.399925 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:11:18.399933 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:11:18.400157 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:18.400828 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:11:18.400851 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:11:18.403153 (+ 2302us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:11:18.406178 (+ 3025us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:11:18.406189 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:11:18.406207 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:11:18.406712 (+ 505us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:11:18.407169 (+ 457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:11:18.407173 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:11:18.407182 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:11:18.407398 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:11:18.408009 (+ 611us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:11:18.408016 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:11:18.408456 (+ 440us) client_negotiation.cc:769] Sending connection context
0811 09:11:18.408718 (+ 262us) client_negotiation.cc:240] Negotiation successful
0811 09:11:18.409153 (+ 435us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":554,"thread_start_us":359,"threads_started":1}
I20260811 09:11:18.411942 23009 heartbeater.cc:347] Connected to a master server at 127.28.154.190:36117
I20260811 09:11:18.412308 23009 heartbeater.cc:464] Registering TS with master...
I20260811 09:11:18.413218 23009 heartbeater.cc:511] Master 127.28.154.190:36117 requested a full tablet report, sending...
I20260811 09:11:18.415596 22665 ts_manager.cc:194] Registered new tserver with Master: e04ee1cf144a4460abe550ff84ff1b29 (127.28.154.129:44993)
I20260811 09:11:18.417264 22665 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.154.129@KRBTEST.COM'} at 127.28.154.129:43367
WARNING: no policy specified for kudu/127.28.154.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.154.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.154.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.130.keytab.
Entry for principal kudu/127.28.154.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.130.keytab.
I20260811 09:11:18.461107 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.154.130:0
--local_ip_for_outbound_sockets=127.28.154.130
--webserver_interface=127.28.154.130
--webserver_port=0
--tserver_master_addrs=127.28.154.190:36117
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.130.keytab
--principal=kudu/127.28.154.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:32851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:11:18.816401 23016 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:11:18.816712 23016 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:11:18.816809 23016 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:11:18.824191 23016 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:11:18.824319 23016 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:11:18.824486 23016 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.154.130
I20260811 09:11:18.835168 23016 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:32851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.130.keytab
--principal=kudu/127.28.154.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.154.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.154.130
--webserver_port=0
--tserver_master_addrs=127.28.154.190:36117
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.154.130
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:11:18.836510 23016 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:11:18.837797 23016 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:11:18.851758 23024 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:11:18.851713 23028 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:11:18.851673 23025 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:11:18.852499 23016 server_base.cc:1034] running on GCE node
I20260811 09:11:18.853119 23016 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:11:18.854120 23016 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:11:18.855340 23016 hybrid_clock.cc:648] HybridClock initialized: now 1786439478855289 us; error 46 us; skew 500 ppm
I20260811 09:11:18.855687 23016 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:11:18 dist-test-slave-xt4k krb5kdc[22586](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439478, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:11:18.858881 23016 init.cc:377] Logged in from keytab as kudu/127.28.154.130@KRBTEST.COM (short username kudu)
I20260811 09:11:18.860433 23016 webserver.cc:460] Webserver started at http://127.28.154.130:34149/ using document root <none> and password file <none>
I20260811 09:11:18.861153 23016 fs_manager.cc:359] Metadata directory not provided
I20260811 09:11:18.861263 23016 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:11:18.861572 23016 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:11:18.864636 23016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-1/data/instance:
uuid: "048c18609b164ee793f0e33a9ac111a7"
format_stamp: "Formatted at 2026-08-11 09:11:18 on dist-test-slave-xt4k"
server_key: "63e41ac7b26e1cd21bd24dd5da3362e8"
server_key_iv: "a4abc8b6a2eefce27f8af3025ab6be4d"
server_key_version: "encryptionkey@0"
I20260811 09:11:18.865418 23016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-1/wal/instance:
uuid: "048c18609b164ee793f0e33a9ac111a7"
format_stamp: "Formatted at 2026-08-11 09:11:18 on dist-test-slave-xt4k"
server_key: "63e41ac7b26e1cd21bd24dd5da3362e8"
server_key_iv: "a4abc8b6a2eefce27f8af3025ab6be4d"
server_key_version: "encryptionkey@0"
I20260811 09:11:18.870838 23016 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 09:11:18.874691 23040 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:11:18.876250 23016 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 09:11:18.876466 23016 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-1/wal
uuid: "048c18609b164ee793f0e33a9ac111a7"
format_stamp: "Formatted at 2026-08-11 09:11:18 on dist-test-slave-xt4k"
server_key: "63e41ac7b26e1cd21bd24dd5da3362e8"
server_key_iv: "a4abc8b6a2eefce27f8af3025ab6be4d"
server_key_version: "encryptionkey@0"
I20260811 09:11:18.876662 23016 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:11:18.891095 23016 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:11:18.895231 23016 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:11:18.895524 23016 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:11:18.896564 23016 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:11:18.897917 23016 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:11:18.897991 23016 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:11:18.898118 23016 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:11:18.898171 23016 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:11:18.929699 23016 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.130:41151
I20260811 09:11:18.929742 23247 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.130:41151 every 8 connection(s)
I20260811 09:11:18.930990 23016 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-1/data/info.pb
I20260811 09:11:18.941430 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 23016
I20260811 09:11:18.941589 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-1/wal/instance
I20260811 09:11:18.942101 29290 external_mini_cluster.cc:1428] Setting key 49ce30ed984436f831f867fff01948c2
Aug 11 09:11:18 dist-test-slave-xt4k krb5kdc[22586](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439478, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.130@KRBTEST.COM for kudu/127.28.154.190@KRBTEST.COM
I20260811 09:11:18.951635 23253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:18.933660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.130:37633 (local address 127.28.154.190:36117)
0811 09:11:18.934118 (+ 458us) server_negotiation.cc:206] Beginning negotiation
0811 09:11:18.934123 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:11:18.935113 (+ 990us) server_negotiation.cc:407] Connection header received
0811 09:11:18.937021 (+ 1908us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:11:18.937026 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:11:18.937112 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:11:18.937210 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:11:18.938772 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:18.939546 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:18.941269 (+ 1723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:18.941807 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:18.941918 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:18.945983 (+ 4065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:11:18.945988 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:11:18.945992 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:11:18.946024 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:11:18.948547 (+ 2523us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:11:18.949253 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:11:18.949257 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:11:18.949259 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:11:18.949380 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:11:18.949811 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:11:18.949815 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:11:18.949817 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:11:18.950101 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:11:18.950236 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0811 09:11:18.951059 (+ 823us) server_negotiation.cc:299] Negotiation successful
0811 09:11:18.951285 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":226,"threads_started":1}
I20260811 09:11:18.952654 23254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:18.934129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:36117 (local address 127.28.154.130:37633)
0811 09:11:18.934925 (+ 796us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:18.934979 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 09:11:18.936747 (+ 1768us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:11:18.937458 (+ 711us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:11:18.937466 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:11:18.937996 (+ 530us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:11:18.938589 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:11:18.938609 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:11:18.939694 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:11:18.939698 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:11:18.941066 (+ 1368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:11:18.941083 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:11:18.942065 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:11:18.942072 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:11:18.942405 (+ 333us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:18.943241 (+ 836us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:11:18.943273 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:11:18.945750 (+ 2477us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:11:18.948694 (+ 2944us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:11:18.948703 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:11:18.948721 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:11:18.949127 (+ 406us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:11:18.949488 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:11:18.949493 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:11:18.949497 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:11:18.949692 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:11:18.950229 (+ 537us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:11:18.950236 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:11:18.950647 (+ 411us) client_negotiation.cc:769] Sending connection context
0811 09:11:18.950905 (+ 258us) client_negotiation.cc:240] Negotiation successful
0811 09:11:18.951408 (+ 503us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":293,"threads_started":1}
I20260811 09:11:18.954346 23249 heartbeater.cc:347] Connected to a master server at 127.28.154.190:36117
I20260811 09:11:18.954671 23249 heartbeater.cc:464] Registering TS with master...
I20260811 09:11:18.955466 23249 heartbeater.cc:511] Master 127.28.154.190:36117 requested a full tablet report, sending...
I20260811 09:11:18.957079 22665 ts_manager.cc:194] Registered new tserver with Master: 048c18609b164ee793f0e33a9ac111a7 (127.28.154.130:41151)
I20260811 09:11:18.957943 22665 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.154.130@KRBTEST.COM'} at 127.28.154.130:37633
WARNING: no policy specified for kudu/127.28.154.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.154.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.154.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.131.keytab.
Entry for principal kudu/127.28.154.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.131.keytab.
I20260811 09:11:19.007849 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.154.131:0
--local_ip_for_outbound_sockets=127.28.154.131
--webserver_interface=127.28.154.131
--webserver_port=0
--tserver_master_addrs=127.28.154.190:36117
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.131.keytab
--principal=kudu/127.28.154.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:32851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:11:19.352370 23257 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:11:19.352687 23257 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:11:19.352826 23257 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:11:19.360164 23257 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:11:19.360296 23257 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:11:19.360466 23257 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.154.131
I20260811 09:11:19.370880 23257 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:32851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.131.keytab
--principal=kudu/127.28.154.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.154.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.154.131
--webserver_port=0
--tserver_master_addrs=127.28.154.190:36117
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.154.131
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:11:19.372267 23257 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:11:19.373624 23257 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:11:19.387254 23263 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:11:19.387928 23264 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:11:19.388031 23257 server_base.cc:1034] running on GCE node
W20260811 09:11:19.388427 23267 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:11:19.389089 23257 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:11:19.390110 23257 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:11:19.391348 23257 hybrid_clock.cc:648] HybridClock initialized: now 1786439479391281 us; error 56 us; skew 500 ppm
I20260811 09:11:19.391708 23257 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:11:19 dist-test-slave-xt4k krb5kdc[22586](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439479, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:11:19.395033 23257 init.cc:377] Logged in from keytab as kudu/127.28.154.131@KRBTEST.COM (short username kudu)
I20260811 09:11:19.396715 23257 webserver.cc:460] Webserver started at http://127.28.154.131:39891/ using document root <none> and password file <none>
I20260811 09:11:19.397428 23257 fs_manager.cc:359] Metadata directory not provided
I20260811 09:11:19.397559 23257 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:11:19.397853 23257 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:11:19.401142 23257 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-2/data/instance:
uuid: "699b1810d51a439684efddb537a4348e"
format_stamp: "Formatted at 2026-08-11 09:11:19 on dist-test-slave-xt4k"
server_key: "b375fe65da0c0f7e44fafa54c751e21c"
server_key_iv: "2fbd477725914d6faa1482a07ac32eb2"
server_key_version: "encryptionkey@0"
I20260811 09:11:19.401952 23257 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-2/wal/instance:
uuid: "699b1810d51a439684efddb537a4348e"
format_stamp: "Formatted at 2026-08-11 09:11:19 on dist-test-slave-xt4k"
server_key: "b375fe65da0c0f7e44fafa54c751e21c"
server_key_iv: "2fbd477725914d6faa1482a07ac32eb2"
server_key_version: "encryptionkey@0"
I20260811 09:11:19.407300 23257 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 09:11:19.411041 23279 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:11:19.412655 23257 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 09:11:19.412839 23257 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-2/wal
uuid: "699b1810d51a439684efddb537a4348e"
format_stamp: "Formatted at 2026-08-11 09:11:19 on dist-test-slave-xt4k"
server_key: "b375fe65da0c0f7e44fafa54c751e21c"
server_key_iv: "2fbd477725914d6faa1482a07ac32eb2"
server_key_version: "encryptionkey@0"
I20260811 09:11:19.413050 23257 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:11:19.421000 23009 heartbeater.cc:503] Master 127.28.154.190:36117 was elected leader, sending a full tablet report...
I20260811 09:11:19.426918 23257 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:11:19.431156 23257 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:11:19.431450 23257 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:11:19.432653 23257 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:11:19.434123 23257 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:11:19.434207 23257 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:11:19.434334 23257 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:11:19.434386 23257 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:11:19.469228 23257 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.131:35775
I20260811 09:11:19.469259 23482 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.131:35775 every 8 connection(s)
I20260811 09:11:19.470594 23257 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-2/data/info.pb
I20260811 09:11:19.477515 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 23257
I20260811 09:11:19.477718 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-2/wal/instance
I20260811 09:11:19.478178 29290 external_mini_cluster.cc:1428] Setting key 995fd44ff02625546ed0d07eed7bc836
Aug 11 09:11:19 dist-test-slave-xt4k krb5kdc[22586](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439479, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.131@KRBTEST.COM for kudu/127.28.154.190@KRBTEST.COM
I20260811 09:11:19.492302 23487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:19.473286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.131:55595 (local address 127.28.154.190:36117)
0811 09:11:19.473772 (+ 486us) server_negotiation.cc:206] Beginning negotiation
0811 09:11:19.473776 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:11:19.474665 (+ 889us) server_negotiation.cc:407] Connection header received
0811 09:11:19.476562 (+ 1897us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:11:19.476568 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:11:19.476658 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:11:19.476888 (+ 230us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:11:19.478947 (+ 2059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:19.479894 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:19.482002 (+ 2108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:19.482415 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:19.482528 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:19.486297 (+ 3769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:11:19.486303 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:11:19.486306 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:11:19.486341 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:11:19.489115 (+ 2774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:11:19.489907 (+ 792us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:11:19.489911 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:11:19.489913 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:11:19.490044 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:11:19.490499 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:11:19.490502 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:11:19.490504 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:11:19.490784 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:11:19.490934 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0811 09:11:19.491685 (+ 751us) server_negotiation.cc:299] Negotiation successful
0811 09:11:19.491923 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":381,"thread_start_us":252,"threads_started":1}
I20260811 09:11:19.493359 23488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:19.473720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:36117 (local address 127.28.154.131:55595)
0811 09:11:19.474485 (+ 765us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:19.474532 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 09:11:19.476243 (+ 1711us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:11:19.477298 (+ 1055us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:11:19.477308 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:11:19.478041 (+ 733us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:11:19.478757 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:11:19.478779 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:11:19.480051 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:11:19.480056 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:11:19.481857 (+ 1801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:11:19.481871 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:11:19.482653 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:11:19.482661 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:11:19.483004 (+ 343us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:19.483646 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:11:19.483678 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:11:19.486061 (+ 2383us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:11:19.489283 (+ 3222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:11:19.489295 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:11:19.489313 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:11:19.489767 (+ 454us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:11:19.490162 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:11:19.490167 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:11:19.490171 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:11:19.490368 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:11:19.490925 (+ 557us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:11:19.490943 (+ 18us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:11:19.491295 (+ 352us) client_negotiation.cc:769] Sending connection context
0811 09:11:19.491542 (+ 247us) client_negotiation.cc:240] Negotiation successful
0811 09:11:19.492120 (+ 578us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":523,"thread_start_us":299,"threads_started":1}
I20260811 09:11:19.494807 23484 heartbeater.cc:347] Connected to a master server at 127.28.154.190:36117
I20260811 09:11:19.495124 23484 heartbeater.cc:464] Registering TS with master...
I20260811 09:11:19.495901 23484 heartbeater.cc:511] Master 127.28.154.190:36117 requested a full tablet report, sending...
I20260811 09:11:19.497394 22665 ts_manager.cc:194] Registered new tserver with Master: 699b1810d51a439684efddb537a4348e (127.28.154.131:35775)
I20260811 09:11:19.498206 22665 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.154.131@KRBTEST.COM'} at 127.28.154.131:55595
I20260811 09:11:19.506999 29290 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 09:11:19.525861 23487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:19.512001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58694 (local address 127.28.154.190:36117)
0811 09:11:19.512273 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0811 09:11:19.512280 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:11:19.512565 (+ 285us) server_negotiation.cc:407] Connection header received
0811 09:11:19.512705 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:11:19.512709 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:11:19.512780 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:11:19.512862 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:11:19.514327 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:19.515215 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:19.516790 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:19.517202 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:19.517314 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:19.519893 (+ 2579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:11:19.519901 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:11:19.519906 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:11:19.519951 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:11:19.522645 (+ 2694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:11:19.523589 (+ 944us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:11:19.523594 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:11:19.523596 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:11:19.523738 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:11:19.524144 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:11:19.524149 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:11:19.524151 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:11:19.524629 (+ 478us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:11:19.524841 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0811 09:11:19.525216 (+ 375us) server_negotiation.cc:299] Negotiation successful
0811 09:11:19.525417 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108}
I20260811 09:11:19.530400 22665 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58694:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 09:11:19.537550 22665 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 09:11:19.564654 23509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:19.551214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.131:35775 (local address 127.0.0.1:40780)
0811 09:11:19.552515 (+ 1301us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:19.552574 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 09:11:19.552658 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:11:19.556444 (+ 3786us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:11:19.556454 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:11:19.556493 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:11:19.557081 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:11:19.557097 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:11:19.559162 (+ 2065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:11:19.559167 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:11:19.561806 (+ 2639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:11:19.561828 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:11:19.563556 (+ 1728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:11:19.563563 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:11:19.563764 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:19.563823 (+ 59us) client_negotiation.cc:769] Sending connection context
0811 09:11:19.563953 (+ 130us) client_negotiation.cc:240] Negotiation successful
0811 09:11:19.564159 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1207,"mutex_wait_us":144,"thread_start_us":1051,"threads_started":1}
I20260811 09:11:19.565199 23512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:19.551427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56994 (local address 127.28.154.130:41151)
0811 09:11:19.556011 (+ 4584us) server_negotiation.cc:206] Beginning negotiation
0811 09:11:19.556023 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 09:11:19.556072 (+ 49us) server_negotiation.cc:407] Connection header received
0811 09:11:19.556183 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:11:19.556191 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:11:19.556489 (+ 298us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:11:19.556723 (+ 234us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:11:19.557408 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:19.558982 (+ 1574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:19.561822 (+ 2840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:19.563323 (+ 1501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:19.563497 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:19.563708 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0811 09:11:19.564494 (+ 786us) server_negotiation.cc:299] Negotiation successful
0811 09:11:19.564690 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4341,"thread_start_us":4218,"threads_started":1}
I20260811 09:11:19.565307 23513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:19.551857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40780 (local address 127.28.154.131:35775)
0811 09:11:19.553529 (+ 1672us) server_negotiation.cc:206] Beginning negotiation
0811 09:11:19.553540 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 09:11:19.553578 (+ 38us) server_negotiation.cc:407] Connection header received
0811 09:11:19.553690 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:11:19.553699 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:11:19.553977 (+ 278us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:11:19.554199 (+ 222us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:11:19.557284 (+ 3085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:19.559007 (+ 1723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:19.561989 (+ 2982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:19.563349 (+ 1360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:19.564632 (+ 1283us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:19.564805 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0811 09:11:19.564912 (+ 107us) server_negotiation.cc:299] Negotiation successful
0811 09:11:19.565060 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1501,"thread_start_us":1342,"threads_started":1}
I20260811 09:11:19.565649 23507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:19.550938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.130:41151 (local address 127.0.0.1:56994)
0811 09:11:19.551798 (+ 860us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:19.551843 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 09:11:19.552036 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:11:19.556727 (+ 4691us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:11:19.556732 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:11:19.556756 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:11:19.557211 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:11:19.557226 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:11:19.559160 (+ 1934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:11:19.559165 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:11:19.561634 (+ 2469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:11:19.561651 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:11:19.563862 (+ 2211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:11:19.563868 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:11:19.564037 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:19.564067 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 09:11:19.564143 (+ 76us) client_negotiation.cc:240] Negotiation successful
0811 09:11:19.564250 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":592,"mutex_wait_us":238,"thread_start_us":1766,"threads_started":1}
I20260811 09:11:19.567051 23514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:19.553226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.129:44993 (local address 127.0.0.1:38116)
0811 09:11:19.556387 (+ 3161us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:19.556406 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 09:11:19.556530 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:11:19.558499 (+ 1969us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:11:19.558504 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:11:19.558531 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:11:19.558979 (+ 448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:11:19.558994 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:11:19.560751 (+ 1757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:11:19.560757 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:11:19.562776 (+ 2019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:11:19.562792 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:11:19.566005 (+ 3213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:11:19.566012 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:11:19.566320 (+ 308us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:19.566359 (+ 39us) client_negotiation.cc:769] Sending connection context
0811 09:11:19.566473 (+ 114us) client_negotiation.cc:240] Negotiation successful
0811 09:11:19.566615 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3016,"mutex_wait_us":53,"spinlock_wait_cycles":3328,"thread_start_us":2882,"threads_started":1}
I20260811 09:11:19.567155 23520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:19.553618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38116 (local address 127.28.154.129:44993)
0811 09:11:19.557755 (+ 4137us) server_negotiation.cc:206] Beginning negotiation
0811 09:11:19.557762 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:11:19.557784 (+ 22us) server_negotiation.cc:407] Connection header received
0811 09:11:19.557867 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:11:19.557873 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:11:19.558053 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:11:19.558244 (+ 191us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:11:19.559186 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:19.560591 (+ 1405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:19.562965 (+ 2374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:19.565801 (+ 2836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:19.566021 (+ 220us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:19.566178 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0811 09:11:19.566539 (+ 361us) server_negotiation.cc:299] Negotiation successful
0811 09:11:19.566717 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3986,"thread_start_us":3819,"threads_started":1}
I20260811 09:11:19.569753 23367 tablet_service.cc:1467] Processing CreateTablet for tablet d7e17113bd194d0e99f4690fd7593dea (DEFAULT_TABLE table=test-table [id=c47a8c2c3e2d4a7681d1458c26011ead]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:11:19.569834 23129 tablet_service.cc:1467] Processing CreateTablet for tablet d7e17113bd194d0e99f4690fd7593dea (DEFAULT_TABLE table=test-table [id=c47a8c2c3e2d4a7681d1458c26011ead]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:11:19.571113 23367 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d7e17113bd194d0e99f4690fd7593dea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:11:19.571290 23129 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d7e17113bd194d0e99f4690fd7593dea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:11:19.571316 22881 tablet_service.cc:1467] Processing CreateTablet for tablet d7e17113bd194d0e99f4690fd7593dea (DEFAULT_TABLE table=test-table [id=c47a8c2c3e2d4a7681d1458c26011ead]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:11:19.572760 22881 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d7e17113bd194d0e99f4690fd7593dea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:11:19.584789 23521 tablet_bootstrap.cc:492] T d7e17113bd194d0e99f4690fd7593dea P 699b1810d51a439684efddb537a4348e: Bootstrap starting.
I20260811 09:11:19.587039 23522 tablet_bootstrap.cc:492] T d7e17113bd194d0e99f4690fd7593dea P 048c18609b164ee793f0e33a9ac111a7: Bootstrap starting.
I20260811 09:11:19.588667 23525 tablet_bootstrap.cc:492] T d7e17113bd194d0e99f4690fd7593dea P e04ee1cf144a4460abe550ff84ff1b29: Bootstrap starting.
I20260811 09:11:19.588896 23521 tablet_bootstrap.cc:654] T d7e17113bd194d0e99f4690fd7593dea P 699b1810d51a439684efddb537a4348e: Neither blocks nor log segments found. Creating new log.
I20260811 09:11:19.590318 23521 log.cc:824] T d7e17113bd194d0e99f4690fd7593dea P 699b1810d51a439684efddb537a4348e: Log is configured to *not* fsync() on all Append() calls
I20260811 09:11:19.591096 23522 tablet_bootstrap.cc:654] T d7e17113bd194d0e99f4690fd7593dea P 048c18609b164ee793f0e33a9ac111a7: Neither blocks nor log segments found. Creating new log.
I20260811 09:11:19.591570 23525 tablet_bootstrap.cc:654] T d7e17113bd194d0e99f4690fd7593dea P e04ee1cf144a4460abe550ff84ff1b29: Neither blocks nor log segments found. Creating new log.
I20260811 09:11:19.592312 23522 log.cc:824] T d7e17113bd194d0e99f4690fd7593dea P 048c18609b164ee793f0e33a9ac111a7: Log is configured to *not* fsync() on all Append() calls
I20260811 09:11:19.592711 23525 log.cc:824] T d7e17113bd194d0e99f4690fd7593dea P e04ee1cf144a4460abe550ff84ff1b29: Log is configured to *not* fsync() on all Append() calls
I20260811 09:11:19.594383 23521 tablet_bootstrap.cc:492] T d7e17113bd194d0e99f4690fd7593dea P 699b1810d51a439684efddb537a4348e: No bootstrap required, opened a new log
I20260811 09:11:19.594678 23521 ts_tablet_manager.cc:1397] T d7e17113bd194d0e99f4690fd7593dea P 699b1810d51a439684efddb537a4348e: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.003s
I20260811 09:11:19.595471 23522 tablet_bootstrap.cc:492] T d7e17113bd194d0e99f4690fd7593dea P 048c18609b164ee793f0e33a9ac111a7: No bootstrap required, opened a new log
I20260811 09:11:19.595703 23522 ts_tablet_manager.cc:1397] T d7e17113bd194d0e99f4690fd7593dea P 048c18609b164ee793f0e33a9ac111a7: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.004s
I20260811 09:11:19.595767 23525 tablet_bootstrap.cc:492] T d7e17113bd194d0e99f4690fd7593dea P e04ee1cf144a4460abe550ff84ff1b29: No bootstrap required, opened a new log
I20260811 09:11:19.596035 23525 ts_tablet_manager.cc:1397] T d7e17113bd194d0e99f4690fd7593dea P e04ee1cf144a4460abe550ff84ff1b29: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.004s
I20260811 09:11:19.602779 23522 raft_consensus.cc:348] T d7e17113bd194d0e99f4690fd7593dea P 048c18609b164ee793f0e33a9ac111a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "048c18609b164ee793f0e33a9ac111a7" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 41151 } } peers { permanent_uuid: "699b1810d51a439684efddb537a4348e" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 35775 } } peers { permanent_uuid: "e04ee1cf144a4460abe550ff84ff1b29" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 44993 } }
I20260811 09:11:19.603351 23522 raft_consensus.cc:374] T d7e17113bd194d0e99f4690fd7593dea P 048c18609b164ee793f0e33a9ac111a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:11:19.603480 23522 raft_consensus.cc:729] T d7e17113bd194d0e99f4690fd7593dea P 048c18609b164ee793f0e33a9ac111a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 048c18609b164ee793f0e33a9ac111a7, State: Initialized, Role: FOLLOWER
I20260811 09:11:19.604384 23522 consensus_queue.cc:260] T d7e17113bd194d0e99f4690fd7593dea P 048c18609b164ee793f0e33a9ac111a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "048c18609b164ee793f0e33a9ac111a7" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 41151 } } peers { permanent_uuid: "699b1810d51a439684efddb537a4348e" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 35775 } } peers { permanent_uuid: "e04ee1cf144a4460abe550ff84ff1b29" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 44993 } }
I20260811 09:11:19.605767 23249 heartbeater.cc:503] Master 127.28.154.190:36117 was elected leader, sending a full tablet report...
I20260811 09:11:19.605767 23521 raft_consensus.cc:348] T d7e17113bd194d0e99f4690fd7593dea P 699b1810d51a439684efddb537a4348e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "048c18609b164ee793f0e33a9ac111a7" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 41151 } } peers { permanent_uuid: "699b1810d51a439684efddb537a4348e" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 35775 } } peers { permanent_uuid: "e04ee1cf144a4460abe550ff84ff1b29" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 44993 } }
I20260811 09:11:19.606405 23521 raft_consensus.cc:374] T d7e17113bd194d0e99f4690fd7593dea P 699b1810d51a439684efddb537a4348e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:11:19.606510 23521 raft_consensus.cc:729] T d7e17113bd194d0e99f4690fd7593dea P 699b1810d51a439684efddb537a4348e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 699b1810d51a439684efddb537a4348e, State: Initialized, Role: FOLLOWER
I20260811 09:11:19.606570 23522 ts_tablet_manager.cc:1428] T d7e17113bd194d0e99f4690fd7593dea P 048c18609b164ee793f0e33a9ac111a7: Time spent starting tablet: real 0.011s user 0.007s sys 0.004s
I20260811 09:11:19.606632 23525 raft_consensus.cc:348] T d7e17113bd194d0e99f4690fd7593dea P e04ee1cf144a4460abe550ff84ff1b29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "048c18609b164ee793f0e33a9ac111a7" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 41151 } } peers { permanent_uuid: "699b1810d51a439684efddb537a4348e" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 35775 } } peers { permanent_uuid: "e04ee1cf144a4460abe550ff84ff1b29" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 44993 } }
I20260811 09:11:19.607141 23525 raft_consensus.cc:374] T d7e17113bd194d0e99f4690fd7593dea P e04ee1cf144a4460abe550ff84ff1b29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:11:19.607237 23525 raft_consensus.cc:729] T d7e17113bd194d0e99f4690fd7593dea P e04ee1cf144a4460abe550ff84ff1b29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e04ee1cf144a4460abe550ff84ff1b29, State: Initialized, Role: FOLLOWER
I20260811 09:11:19.607396 23521 consensus_queue.cc:260] T d7e17113bd194d0e99f4690fd7593dea P 699b1810d51a439684efddb537a4348e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "048c18609b164ee793f0e33a9ac111a7" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 41151 } } peers { permanent_uuid: "699b1810d51a439684efddb537a4348e" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 35775 } } peers { permanent_uuid: "e04ee1cf144a4460abe550ff84ff1b29" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 44993 } }
I20260811 09:11:19.607841 23525 consensus_queue.cc:260] T d7e17113bd194d0e99f4690fd7593dea P e04ee1cf144a4460abe550ff84ff1b29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "048c18609b164ee793f0e33a9ac111a7" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 41151 } } peers { permanent_uuid: "699b1810d51a439684efddb537a4348e" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 35775 } } peers { permanent_uuid: "e04ee1cf144a4460abe550ff84ff1b29" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 44993 } }
I20260811 09:11:19.608633 23484 heartbeater.cc:503] Master 127.28.154.190:36117 was elected leader, sending a full tablet report...
I20260811 09:11:19.610060 23521 ts_tablet_manager.cc:1428] T d7e17113bd194d0e99f4690fd7593dea P 699b1810d51a439684efddb537a4348e: Time spent starting tablet: real 0.015s user 0.015s sys 0.000s
I20260811 09:11:19.612296 23525 ts_tablet_manager.cc:1428] T d7e17113bd194d0e99f4690fd7593dea P e04ee1cf144a4460abe550ff84ff1b29: Time spent starting tablet: real 0.016s user 0.007s sys 0.008s
W20260811 09:11:19.641119 23010 tablet.cc:2369] T d7e17113bd194d0e99f4690fd7593dea P e04ee1cf144a4460abe550ff84ff1b29: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 09:11:19.683018 23251 tablet.cc:2369] T d7e17113bd194d0e99f4690fd7593dea P 048c18609b164ee793f0e33a9ac111a7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 09:11:19.722292 23485 tablet.cc:2369] T d7e17113bd194d0e99f4690fd7593dea P 699b1810d51a439684efddb537a4348e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 09:11:19.875244 23530 raft_consensus.cc:482] T d7e17113bd194d0e99f4690fd7593dea P 048c18609b164ee793f0e33a9ac111a7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:11:19.875556 23530 raft_consensus.cc:504] T d7e17113bd194d0e99f4690fd7593dea P 048c18609b164ee793f0e33a9ac111a7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "048c18609b164ee793f0e33a9ac111a7" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 41151 } } peers { permanent_uuid: "699b1810d51a439684efddb537a4348e" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 35775 } } peers { permanent_uuid: "e04ee1cf144a4460abe550ff84ff1b29" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 44993 } }
I20260811 09:11:19.877391 23530 leader_election.cc:288] T d7e17113bd194d0e99f4690fd7593dea P 048c18609b164ee793f0e33a9ac111a7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 699b1810d51a439684efddb537a4348e (127.28.154.131:35775), e04ee1cf144a4460abe550ff84ff1b29 (127.28.154.129:44993)
I20260811 09:11:19.886498 23513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:19.878022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.130:55513 (local address 127.28.154.131:35775)
0811 09:11:19.878234 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0811 09:11:19.878240 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:11:19.878828 (+ 588us) server_negotiation.cc:407] Connection header received
0811 09:11:19.879042 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:11:19.879046 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:11:19.879114 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:11:19.879210 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:11:19.879900 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:19.881111 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:19.884150 (+ 3039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:19.885425 (+ 1275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:19.885579 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:19.885658 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 09:11:19.886116 (+ 458us) server_negotiation.cc:299] Negotiation successful
0811 09:11:19.886241 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 09:11:19.886500 23520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:19.878110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.130:55347 (local address 127.28.154.129:44993)
0811 09:11:19.878428 (+ 318us) server_negotiation.cc:206] Beginning negotiation
0811 09:11:19.878436 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:11:19.879143 (+ 707us) server_negotiation.cc:407] Connection header received
0811 09:11:19.879293 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:11:19.879298 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:11:19.879403 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:11:19.879472 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:11:19.880146 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:19.881241 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:19.884150 (+ 2909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:19.885406 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:19.885580 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:19.885658 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 09:11:19.886100 (+ 442us) server_negotiation.cc:299] Negotiation successful
0811 09:11:19.886241 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160}
I20260811 09:11:19.886626 23533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:19.877892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.129:44993 (local address 127.28.154.130:55347)
0811 09:11:19.879022 (+ 1130us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:19.879040 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 09:11:19.879164 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:11:19.879525 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:11:19.879530 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:11:19.879553 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:11:19.879986 (+ 433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:11:19.880000 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:11:19.881364 (+ 1364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:11:19.881369 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:11:19.883960 (+ 2591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:11:19.883973 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:11:19.885673 (+ 1700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:11:19.885678 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:11:19.885911 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:19.885975 (+ 64us) client_negotiation.cc:769] Sending connection context
0811 09:11:19.886063 (+ 88us) client_negotiation.cc:240] Negotiation successful
0811 09:11:19.886197 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1030,"mutex_wait_us":197,"thread_start_us":436,"threads_started":1}
I20260811 09:11:19.887050 23535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:19.877753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.131:35775 (local address 127.28.154.130:55513)
0811 09:11:19.878695 (+ 942us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:19.878712 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 09:11:19.878888 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:11:19.879254 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:11:19.879259 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:11:19.879284 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:11:19.879730 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:11:19.879746 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:11:19.881278 (+ 1532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:11:19.881284 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:11:19.883956 (+ 2672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:11:19.883973 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:11:19.885650 (+ 1677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:11:19.885661 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:11:19.885915 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:19.885963 (+ 48us) client_negotiation.cc:769] Sending connection context
0811 09:11:19.886063 (+ 100us) client_negotiation.cc:240] Negotiation successful
0811 09:11:19.886579 (+ 516us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":796,"mutex_wait_us":215,"thread_start_us":968,"threads_started":1}
I20260811 09:11:19.887794 22913 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7e17113bd194d0e99f4690fd7593dea" candidate_uuid: "048c18609b164ee793f0e33a9ac111a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e04ee1cf144a4460abe550ff84ff1b29" is_pre_election: true
I20260811 09:11:19.887948 23402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7e17113bd194d0e99f4690fd7593dea" candidate_uuid: "048c18609b164ee793f0e33a9ac111a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "699b1810d51a439684efddb537a4348e" is_pre_election: true
I20260811 09:11:19.888329 22913 raft_consensus.cc:2454] T d7e17113bd194d0e99f4690fd7593dea P e04ee1cf144a4460abe550ff84ff1b29 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 048c18609b164ee793f0e33a9ac111a7 in term 0.
I20260811 09:11:19.888487 23402 raft_consensus.cc:2454] T d7e17113bd194d0e99f4690fd7593dea P 699b1810d51a439684efddb537a4348e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 048c18609b164ee793f0e33a9ac111a7 in term 0.
I20260811 09:11:19.889119 23043 leader_election.cc:302] T d7e17113bd194d0e99f4690fd7593dea P 048c18609b164ee793f0e33a9ac111a7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 048c18609b164ee793f0e33a9ac111a7, 699b1810d51a439684efddb537a4348e, e04ee1cf144a4460abe550ff84ff1b29; no voters:
I20260811 09:11:19.889515 23530 raft_consensus.cc:2781] T d7e17113bd194d0e99f4690fd7593dea P 048c18609b164ee793f0e33a9ac111a7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 09:11:19.889664 23530 raft_consensus.cc:482] T d7e17113bd194d0e99f4690fd7593dea P 048c18609b164ee793f0e33a9ac111a7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 09:11:19.889756 23530 raft_consensus.cc:3037] T d7e17113bd194d0e99f4690fd7593dea P 048c18609b164ee793f0e33a9ac111a7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:11:19.892129 23530 raft_consensus.cc:504] T d7e17113bd194d0e99f4690fd7593dea P 048c18609b164ee793f0e33a9ac111a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "048c18609b164ee793f0e33a9ac111a7" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 41151 } } peers { permanent_uuid: "699b1810d51a439684efddb537a4348e" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 35775 } } peers { permanent_uuid: "e04ee1cf144a4460abe550ff84ff1b29" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 44993 } }
I20260811 09:11:19.892831 23530 leader_election.cc:288] T d7e17113bd194d0e99f4690fd7593dea P 048c18609b164ee793f0e33a9ac111a7 [CANDIDATE]: Term 1 election: Requested vote from peers 699b1810d51a439684efddb537a4348e (127.28.154.131:35775), e04ee1cf144a4460abe550ff84ff1b29 (127.28.154.129:44993)
I20260811 09:11:19.893362 23402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7e17113bd194d0e99f4690fd7593dea" candidate_uuid: "048c18609b164ee793f0e33a9ac111a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "699b1810d51a439684efddb537a4348e"
I20260811 09:11:19.893366 22913 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d7e17113bd194d0e99f4690fd7593dea" candidate_uuid: "048c18609b164ee793f0e33a9ac111a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e04ee1cf144a4460abe550ff84ff1b29"
I20260811 09:11:19.893535 23402 raft_consensus.cc:3037] T d7e17113bd194d0e99f4690fd7593dea P 699b1810d51a439684efddb537a4348e [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:11:19.893571 22913 raft_consensus.cc:3037] T d7e17113bd194d0e99f4690fd7593dea P e04ee1cf144a4460abe550ff84ff1b29 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:11:19.895715 23402 raft_consensus.cc:2454] T d7e17113bd194d0e99f4690fd7593dea P 699b1810d51a439684efddb537a4348e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 048c18609b164ee793f0e33a9ac111a7 in term 1.
I20260811 09:11:19.895730 22913 raft_consensus.cc:2454] T d7e17113bd194d0e99f4690fd7593dea P e04ee1cf144a4460abe550ff84ff1b29 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 048c18609b164ee793f0e33a9ac111a7 in term 1.
I20260811 09:11:19.896106 23043 leader_election.cc:302] T d7e17113bd194d0e99f4690fd7593dea P 048c18609b164ee793f0e33a9ac111a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 048c18609b164ee793f0e33a9ac111a7, e04ee1cf144a4460abe550ff84ff1b29; no voters:
I20260811 09:11:19.896405 23530 raft_consensus.cc:2781] T d7e17113bd194d0e99f4690fd7593dea P 048c18609b164ee793f0e33a9ac111a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:11:19.896899 23530 raft_consensus.cc:686] T d7e17113bd194d0e99f4690fd7593dea P 048c18609b164ee793f0e33a9ac111a7 [term 1 LEADER]: Becoming Leader. State: Replica: 048c18609b164ee793f0e33a9ac111a7, State: Running, Role: LEADER
I20260811 09:11:19.897434 23530 consensus_queue.cc:237] T d7e17113bd194d0e99f4690fd7593dea P 048c18609b164ee793f0e33a9ac111a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "048c18609b164ee793f0e33a9ac111a7" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 41151 } } peers { permanent_uuid: "699b1810d51a439684efddb537a4348e" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 35775 } } peers { permanent_uuid: "e04ee1cf144a4460abe550ff84ff1b29" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 44993 } }
I20260811 09:11:19.901968 22662 catalog_manager.cc:5495] T d7e17113bd194d0e99f4690fd7593dea P 048c18609b164ee793f0e33a9ac111a7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 048c18609b164ee793f0e33a9ac111a7 (127.28.154.130). New cstate: current_term: 1 leader_uuid: "048c18609b164ee793f0e33a9ac111a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "048c18609b164ee793f0e33a9ac111a7" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 41151 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "699b1810d51a439684efddb537a4348e" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 35775 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e04ee1cf144a4460abe550ff84ff1b29" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 44993 } health_report { overall_health: UNKNOWN } } }
I20260811 09:11:19.933468 23512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:19.927370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57004 (local address 127.28.154.130:41151)
0811 09:11:19.927584 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0811 09:11:19.927590 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:11:19.927609 (+ 19us) server_negotiation.cc:407] Connection header received
0811 09:11:19.927829 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:11:19.927833 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:11:19.927922 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:11:19.928060 (+ 138us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:11:19.928530 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:19.929469 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:19.930978 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:19.931641 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:19.931759 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:19.932165 (+ 406us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:11:19.932849 (+ 684us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:11:19.932976 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0811 09:11:19.933100 (+ 124us) server_negotiation.cc:299] Negotiation successful
0811 09:11:19.933185 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 09:11:19.942107 23402 raft_consensus.cc:1261] T d7e17113bd194d0e99f4690fd7593dea P 699b1810d51a439684efddb537a4348e [term 1 FOLLOWER]: Refusing update from remote peer 048c18609b164ee793f0e33a9ac111a7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:11:19.942380 22913 raft_consensus.cc:1261] T d7e17113bd194d0e99f4690fd7593dea P e04ee1cf144a4460abe550ff84ff1b29 [term 1 FOLLOWER]: Refusing update from remote peer 048c18609b164ee793f0e33a9ac111a7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:11:19.943112 23530 consensus_queue.cc:1035] T d7e17113bd194d0e99f4690fd7593dea P 048c18609b164ee793f0e33a9ac111a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "699b1810d51a439684efddb537a4348e" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 35775 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:11:19.943387 23536 consensus_queue.cc:1035] T d7e17113bd194d0e99f4690fd7593dea P 048c18609b164ee793f0e33a9ac111a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e04ee1cf144a4460abe550ff84ff1b29" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 44993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:11:19.955201 23542 mvcc.cc:205] Tried to move back new op lower bound from 7317256109832851456 to 7317256109663563776. Current Snapshot: MvccSnapshot[applied={T|T < 7317256109832851456}]
I20260811 09:11:19.975395 22664 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58694:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:11:19.975752 22664 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58694:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:11:19.980679 22664 catalog_manager.cc:5782] T 00000000000000000000000000000000 P adc54f73b9dc4ef09d9ede64155cd1e5: Sending DeleteTablet for 3 replicas of tablet d7e17113bd194d0e99f4690fd7593dea
I20260811 09:11:19.982183 22881 tablet_service.cc:1514] Processing DeleteTablet for tablet d7e17113bd194d0e99f4690fd7593dea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:11:19 UTC) from {username='kudu', principal='kudu/127.28.154.190@KRBTEST.COM'} at 127.0.0.1:38116
I20260811 09:11:19.982249 23129 tablet_service.cc:1514] Processing DeleteTablet for tablet d7e17113bd194d0e99f4690fd7593dea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:11:19 UTC) from {username='kudu', principal='kudu/127.28.154.190@KRBTEST.COM'} at 127.0.0.1:56994
I20260811 09:11:19.982235 23367 tablet_service.cc:1514] Processing DeleteTablet for tablet d7e17113bd194d0e99f4690fd7593dea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:11:19 UTC) from {username='kudu', principal='kudu/127.28.154.190@KRBTEST.COM'} at 127.0.0.1:40780
I20260811 09:11:19.982806 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 22767
I20260811 09:11:19.994613 23557 tablet_replica.cc:330] T d7e17113bd194d0e99f4690fd7593dea P 699b1810d51a439684efddb537a4348e: stopping tablet replica
I20260811 09:11:19.995294 23559 tablet_replica.cc:330] T d7e17113bd194d0e99f4690fd7593dea P 048c18609b164ee793f0e33a9ac111a7: stopping tablet replica
I20260811 09:11:19.995865 23559 raft_consensus.cc:2229] T d7e17113bd194d0e99f4690fd7593dea P 048c18609b164ee793f0e33a9ac111a7 [term 1 LEADER]: Raft consensus shutting down.
I20260811 09:11:19.996793 23559 raft_consensus.cc:2258] T d7e17113bd194d0e99f4690fd7593dea P 048c18609b164ee793f0e33a9ac111a7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:11:19.997758 23558 tablet_replica.cc:330] T d7e17113bd194d0e99f4690fd7593dea P e04ee1cf144a4460abe550ff84ff1b29: stopping tablet replica
I20260811 09:11:19.998286 23558 raft_consensus.cc:2229] T d7e17113bd194d0e99f4690fd7593dea P e04ee1cf144a4460abe550ff84ff1b29 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:11:19.998488 23557 raft_consensus.cc:2229] T d7e17113bd194d0e99f4690fd7593dea P 699b1810d51a439684efddb537a4348e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:11:19.998996 23557 raft_consensus.cc:2258] T d7e17113bd194d0e99f4690fd7593dea P 699b1810d51a439684efddb537a4348e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:11:19.999011 23558 raft_consensus.cc:2258] T d7e17113bd194d0e99f4690fd7593dea P e04ee1cf144a4460abe550ff84ff1b29 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:11:20.001454 23558 ts_tablet_manager.cc:1905] T d7e17113bd194d0e99f4690fd7593dea P e04ee1cf144a4460abe550ff84ff1b29: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:11:20.001454 23557 ts_tablet_manager.cc:1905] T d7e17113bd194d0e99f4690fd7593dea P 699b1810d51a439684efddb537a4348e: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 11 09:11:20 dist-test-slave-xt4k krb5kdc[22586](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439477, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.154.129@KRBTEST.COM
I20260811 09:11:20.010020 23558 ts_tablet_manager.cc:1918] T d7e17113bd194d0e99f4690fd7593dea P e04ee1cf144a4460abe550ff84ff1b29: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:11:20.010293 23559 ts_tablet_manager.cc:1905] T d7e17113bd194d0e99f4690fd7593dea P 048c18609b164ee793f0e33a9ac111a7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:11:20.010352 23558 log.cc:1196] T d7e17113bd194d0e99f4690fd7593dea P e04ee1cf144a4460abe550ff84ff1b29: Deleting WAL directory at /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-0/wal/wals/d7e17113bd194d0e99f4690fd7593dea
I20260811 09:11:20.011128 23558 ts_tablet_manager.cc:1939] T d7e17113bd194d0e99f4690fd7593dea P e04ee1cf144a4460abe550ff84ff1b29: Deleting consensus metadata
I20260811 09:11:20.013415 22638 catalog_manager.cc:4841] TS e04ee1cf144a4460abe550ff84ff1b29 (127.28.154.129:44993): tablet d7e17113bd194d0e99f4690fd7593dea (table test-table [id=c47a8c2c3e2d4a7681d1458c26011ead]) successfully deleted
I20260811 09:11:20.015146 23557 ts_tablet_manager.cc:1918] T d7e17113bd194d0e99f4690fd7593dea P 699b1810d51a439684efddb537a4348e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:11:20.015444 23557 log.cc:1196] T d7e17113bd194d0e99f4690fd7593dea P 699b1810d51a439684efddb537a4348e: Deleting WAL directory at /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-2/wal/wals/d7e17113bd194d0e99f4690fd7593dea
I20260811 09:11:20.016069 23557 ts_tablet_manager.cc:1939] T d7e17113bd194d0e99f4690fd7593dea P 699b1810d51a439684efddb537a4348e: Deleting consensus metadata
I20260811 09:11:20.018394 22642 catalog_manager.cc:4841] TS 699b1810d51a439684efddb537a4348e (127.28.154.131:35775): tablet d7e17113bd194d0e99f4690fd7593dea (table test-table [id=c47a8c2c3e2d4a7681d1458c26011ead]) successfully deleted
I20260811 09:11:20.018522 23559 ts_tablet_manager.cc:1918] T d7e17113bd194d0e99f4690fd7593dea P 048c18609b164ee793f0e33a9ac111a7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:11:20.018774 23559 log.cc:1196] T d7e17113bd194d0e99f4690fd7593dea P 048c18609b164ee793f0e33a9ac111a7: Deleting WAL directory at /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-1/wal/wals/d7e17113bd194d0e99f4690fd7593dea
I20260811 09:11:20.019423 23559 ts_tablet_manager.cc:1939] T d7e17113bd194d0e99f4690fd7593dea P 048c18609b164ee793f0e33a9ac111a7: Deleting consensus metadata
I20260811 09:11:20.019464 23520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:19.984021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38130 (local address 127.28.154.129:44993)
0811 09:11:19.984208 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 09:11:19.984215 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:11:19.995022 (+ 10807us) server_negotiation.cc:407] Connection header received
0811 09:11:19.995557 (+ 535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:11:19.995563 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:11:19.995651 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:11:19.995729 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:11:19.998113 (+ 2384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:19.999583 (+ 1470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:20.001778 (+ 2195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:20.002917 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:20.003044 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:20.009908 (+ 6864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:11:20.009927 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:11:20.009949 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:11:20.010012 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:11:20.014531 (+ 4519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:11:20.015757 (+ 1226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:11:20.015769 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:11:20.015773 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:11:20.015967 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:11:20.016676 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:11:20.016683 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:11:20.016686 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:11:20.017440 (+ 754us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:11:20.017791 (+ 351us) server_negotiation.cc:1035] Waiting for connection context
0811 09:11:20.018529 (+ 738us) server_negotiation.cc:299] Negotiation successful
0811 09:11:20.018820 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 09:11:20.020639 22999 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:11:20.021600 22638 catalog_manager.cc:4841] TS 048c18609b164ee793f0e33a9ac111a7 (127.28.154.130:41151): tablet d7e17113bd194d0e99f4690fd7593dea (table test-table [id=c47a8c2c3e2d4a7681d1458c26011ead]) successfully deleted
I20260811 09:11:20.117658 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 22767
I20260811 09:11:20.149509 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 23016
Aug 11 09:11:20 dist-test-slave-xt4k krb5kdc[22586](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439477, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.154.130@KRBTEST.COM
I20260811 09:11:20.173923 23512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:20.150287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57014 (local address 127.28.154.130:41151)
0811 09:11:20.150481 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 09:11:20.150487 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:11:20.150511 (+ 24us) server_negotiation.cc:407] Connection header received
0811 09:11:20.150660 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:11:20.150666 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:11:20.150750 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:11:20.150873 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:11:20.152392 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:20.153608 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:20.155011 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:20.155686 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:20.155824 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:20.165788 (+ 9964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:11:20.165813 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:11:20.165838 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:11:20.165906 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:11:20.170369 (+ 4463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:11:20.171229 (+ 860us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:11:20.171244 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:11:20.171248 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:11:20.171429 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:11:20.171837 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:11:20.171843 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:11:20.171846 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:11:20.172387 (+ 541us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:11:20.172748 (+ 361us) server_negotiation.cc:1035] Waiting for connection context
0811 09:11:20.173249 (+ 501us) server_negotiation.cc:299] Negotiation successful
0811 09:11:20.173508 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 09:11:20.174696 23240 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:11:20.294409 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 23016
I20260811 09:11:20.316470 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 23257
Aug 11 09:11:20 dist-test-slave-xt4k krb5kdc[22586](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439477, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.154.131@KRBTEST.COM
I20260811 09:11:20.333138 23513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:20.317198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40794 (local address 127.28.154.131:35775)
0811 09:11:20.317407 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0811 09:11:20.317412 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:11:20.317487 (+ 75us) server_negotiation.cc:407] Connection header received
0811 09:11:20.317650 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:11:20.317654 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:11:20.317720 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:11:20.317850 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:11:20.319121 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:20.319859 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:20.321250 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:20.321722 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:20.321829 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:20.326705 (+ 4876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:11:20.326727 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:11:20.326746 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:11:20.326794 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:11:20.329850 (+ 3056us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:11:20.330735 (+ 885us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:11:20.330745 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:11:20.330747 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:11:20.330908 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:11:20.331325 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:11:20.331329 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:11:20.331331 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:11:20.331746 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:11:20.332028 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0811 09:11:20.332594 (+ 566us) server_negotiation.cc:299] Negotiation successful
0811 09:11:20.332775 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 09:11:20.333783 23476 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:11:20.426007 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 23257
I20260811 09:11:20.446965 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 22610
I20260811 09:11:20.448189 22715 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:11:20.562399 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 22610
I20260811 09:11:20.598747 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.154.190:36117
--webserver_interface=127.28.154.190
--webserver_port=37189
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab
--principal=kudu/127.28.154.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:32851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.154.190:36117
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:11:20.971248 23572 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:11:20.971554 23572 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:11:20.971693 23572 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:11:20.979115 23572 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:11:20.979240 23572 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:11:20.979300 23572 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:11:20.979388 23572 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:11:20.979444 23572 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:11:20.989802 23572 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:32851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.28.154.190:36117
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab
--principal=kudu/127.28.154.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.154.190:36117
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.154.190
--webserver_port=37189
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:11:20.991209 23572 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:11:20.992894 23572 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:11:21.021572 23589 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:11:21.021641 23582 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:11:21.021920 23583 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:11:21.022555 23572 server_base.cc:1034] running on GCE node
I20260811 09:11:21.023569 23572 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:11:21.025799 23572 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:11:21.027120 23572 hybrid_clock.cc:648] HybridClock initialized: now 1786439481027060 us; error 51 us; skew 500 ppm
I20260811 09:11:21.027657 23572 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:11:21 dist-test-slave-xt4k krb5kdc[22586](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439481, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:11:21.031659 23572 init.cc:377] Logged in from keytab as kudu/127.28.154.190@KRBTEST.COM (short username kudu)
I20260811 09:11:21.033671 23572 webserver.cc:460] Webserver started at http://127.28.154.190:37189/ using document root <none> and password file <none>
I20260811 09:11:21.034518 23572 fs_manager.cc:359] Metadata directory not provided
I20260811 09:11:21.034663 23572 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:11:21.042058 23572 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 09:11:21.045849 23601 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:11:21.047888 23572 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 09:11:21.048127 23572 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/master-0/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/master-0/wal
uuid: "adc54f73b9dc4ef09d9ede64155cd1e5"
format_stamp: "Formatted at 2026-08-11 09:11:17 on dist-test-slave-xt4k"
server_key: "546556857f41fa52df510ea4579ae8bf"
server_key_iv: "56d120bb3d2faf25c7812b93fdb857ed"
server_key_version: "encryptionkey@0"
I20260811 09:11:21.049413 23572 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:11:21.079396 23572 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:11:21.085321 23572 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:11:21.085649 23572 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:11:21.107749 23572 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.190:36117
I20260811 09:11:21.107767 23697 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.190:36117 every 8 connection(s)
I20260811 09:11:21.110003 23572 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
I20260811 09:11:21.117916 23698 sys_catalog.cc:263] Verifying existing consensus state
I20260811 09:11:21.120390 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 23572
I20260811 09:11:21.121490 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.154.129:44993
--local_ip_for_outbound_sockets=127.28.154.129
--tserver_master_addrs=127.28.154.190:36117
--webserver_port=40307
--webserver_interface=127.28.154.129
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.129.keytab
--principal=kudu/127.28.154.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:32851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 09:11:21.122119 23698 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P adc54f73b9dc4ef09d9ede64155cd1e5: Bootstrap starting.
I20260811 09:11:21.143723 23698 log.cc:824] T 00000000000000000000000000000000 P adc54f73b9dc4ef09d9ede64155cd1e5: Log is configured to *not* fsync() on all Append() calls
I20260811 09:11:21.158509 23698 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P adc54f73b9dc4ef09d9ede64155cd1e5: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 09:11:21.159138 23698 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P adc54f73b9dc4ef09d9ede64155cd1e5: Bootstrap complete.
I20260811 09:11:21.168957 23698 raft_consensus.cc:348] T 00000000000000000000000000000000 P adc54f73b9dc4ef09d9ede64155cd1e5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adc54f73b9dc4ef09d9ede64155cd1e5" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 36117 } }
I20260811 09:11:21.169904 23698 raft_consensus.cc:729] T 00000000000000000000000000000000 P adc54f73b9dc4ef09d9ede64155cd1e5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: adc54f73b9dc4ef09d9ede64155cd1e5, State: Initialized, Role: FOLLOWER
I20260811 09:11:21.170584 23698 consensus_queue.cc:260] T 00000000000000000000000000000000 P adc54f73b9dc4ef09d9ede64155cd1e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adc54f73b9dc4ef09d9ede64155cd1e5" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 36117 } }
I20260811 09:11:21.170821 23698 raft_consensus.cc:388] T 00000000000000000000000000000000 P adc54f73b9dc4ef09d9ede64155cd1e5 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:11:21.170945 23698 raft_consensus.cc:482] T 00000000000000000000000000000000 P adc54f73b9dc4ef09d9ede64155cd1e5 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:11:21.171115 23698 raft_consensus.cc:3037] T 00000000000000000000000000000000 P adc54f73b9dc4ef09d9ede64155cd1e5 [term 1 FOLLOWER]: Advancing to term 2
I20260811 09:11:21.175871 23698 raft_consensus.cc:504] T 00000000000000000000000000000000 P adc54f73b9dc4ef09d9ede64155cd1e5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adc54f73b9dc4ef09d9ede64155cd1e5" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 36117 } }
I20260811 09:11:21.176537 23698 leader_election.cc:302] T 00000000000000000000000000000000 P adc54f73b9dc4ef09d9ede64155cd1e5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: adc54f73b9dc4ef09d9ede64155cd1e5; no voters:
I20260811 09:11:21.177188 23698 leader_election.cc:288] T 00000000000000000000000000000000 P adc54f73b9dc4ef09d9ede64155cd1e5 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 09:11:21.177433 23702 raft_consensus.cc:2781] T 00000000000000000000000000000000 P adc54f73b9dc4ef09d9ede64155cd1e5 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 09:11:21.178494 23702 raft_consensus.cc:686] T 00000000000000000000000000000000 P adc54f73b9dc4ef09d9ede64155cd1e5 [term 2 LEADER]: Becoming Leader. State: Replica: adc54f73b9dc4ef09d9ede64155cd1e5, State: Running, Role: LEADER
I20260811 09:11:21.179289 23702 consensus_queue.cc:237] T 00000000000000000000000000000000 P adc54f73b9dc4ef09d9ede64155cd1e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adc54f73b9dc4ef09d9ede64155cd1e5" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 36117 } }
I20260811 09:11:21.179574 23698 sys_catalog.cc:564] T 00000000000000000000000000000000 P adc54f73b9dc4ef09d9ede64155cd1e5 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:11:21.184116 23706 sys_catalog.cc:455] T 00000000000000000000000000000000 P adc54f73b9dc4ef09d9ede64155cd1e5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader adc54f73b9dc4ef09d9ede64155cd1e5. Latest consensus state: current_term: 2 leader_uuid: "adc54f73b9dc4ef09d9ede64155cd1e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adc54f73b9dc4ef09d9ede64155cd1e5" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 36117 } } }
I20260811 09:11:21.184656 23706 sys_catalog.cc:458] T 00000000000000000000000000000000 P adc54f73b9dc4ef09d9ede64155cd1e5 [sys.catalog]: This master's current role is: LEADER
I20260811 09:11:21.184099 23704 sys_catalog.cc:455] T 00000000000000000000000000000000 P adc54f73b9dc4ef09d9ede64155cd1e5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "adc54f73b9dc4ef09d9ede64155cd1e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adc54f73b9dc4ef09d9ede64155cd1e5" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 36117 } } }
I20260811 09:11:21.184878 23704 sys_catalog.cc:458] T 00000000000000000000000000000000 P adc54f73b9dc4ef09d9ede64155cd1e5 [sys.catalog]: This master's current role is: LEADER
I20260811 09:11:21.193956 23717 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:11:21.203236 23717 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:11:21.205060 23717 catalog_manager.cc:1259] Loaded cluster ID: f632f10c6dc34ca093d24d59728d1a6b
I20260811 09:11:21.205261 23717 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:11:21.208884 23717 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260811 09:11:21.227455 23717 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260811 09:11:21.237927 23717 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:11:21.241480 23717 catalog_manager.cc:5879] T 00000000000000000000000000000000 P adc54f73b9dc4ef09d9ede64155cd1e5: Loaded TSK: 0
I20260811 09:11:21.244006 23717 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 09:11:21.514194 23700 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:11:21.514587 23700 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:11:21.514830 23700 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:11:21.522681 23700 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:11:21.522835 23700 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:11:21.522996 23700 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.154.129
I20260811 09:11:21.533706 23700 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:32851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.129.keytab
--principal=kudu/127.28.154.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.154.129:44993
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.154.129
--webserver_port=40307
--tserver_master_addrs=127.28.154.190:36117
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.154.129
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:11:21.535283 23700 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:11:21.536846 23700 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:11:21.553761 23743 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:11:21.553702 23744 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:11:21.553851 23748 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:11:21.554553 23700 server_base.cc:1034] running on GCE node
I20260811 09:11:21.555351 23700 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:11:21.556555 23700 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:11:21.557821 23700 hybrid_clock.cc:648] HybridClock initialized: now 1786439481557744 us; error 64 us; skew 500 ppm
I20260811 09:11:21.558209 23700 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:11:21 dist-test-slave-xt4k krb5kdc[22586](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439481, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:11:21.561836 23700 init.cc:377] Logged in from keytab as kudu/127.28.154.129@KRBTEST.COM (short username kudu)
I20260811 09:11:21.563555 23700 webserver.cc:460] Webserver started at http://127.28.154.129:40307/ using document root <none> and password file <none>
I20260811 09:11:21.564296 23700 fs_manager.cc:359] Metadata directory not provided
I20260811 09:11:21.564484 23700 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:11:21.571410 23700 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.002s sys 0.004s
I20260811 09:11:21.574924 23759 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:11:21.576727 23700 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 09:11:21.576916 23700 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-0/wal
uuid: "e04ee1cf144a4460abe550ff84ff1b29"
format_stamp: "Formatted at 2026-08-11 09:11:18 on dist-test-slave-xt4k"
server_key: "c43fce2f65395065e27c92a5d3d25262"
server_key_iv: "5b5b0ddc94a7f4ad14f8de25489600d0"
server_key_version: "encryptionkey@0"
I20260811 09:11:21.577966 23700 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:11:21.610605 23700 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:11:21.615342 23700 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:11:21.615638 23700 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:11:21.616745 23700 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:11:21.618196 23700 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:11:21.618309 23700 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:11:21.618422 23700 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:11:21.618495 23700 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:11:21.651185 23700 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.129:44993
I20260811 09:11:21.651265 23966 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.129:44993 every 8 connection(s)
I20260811 09:11:21.652787 23700 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-0/data/info.pb
I20260811 09:11:21.653532 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 23700
I20260811 09:11:21.654297 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.154.130:41151
--local_ip_for_outbound_sockets=127.28.154.130
--tserver_master_addrs=127.28.154.190:36117
--webserver_port=34149
--webserver_interface=127.28.154.130
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.130.keytab
--principal=kudu/127.28.154.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:32851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 09:11:21 dist-test-slave-xt4k krb5kdc[22586](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439481, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.129@KRBTEST.COM for kudu/127.28.154.190@KRBTEST.COM
I20260811 09:11:21.681869 23974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:21.656774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:36117 (local address 127.28.154.129:48911)
0811 09:11:21.657689 (+ 915us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:21.657736 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 09:11:21.659486 (+ 1750us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:11:21.661233 (+ 1747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:11:21.661241 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:11:21.661867 (+ 626us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:11:21.662487 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:11:21.662508 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:11:21.664848 (+ 2340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:11:21.664852 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:11:21.666142 (+ 1290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:11:21.666159 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:11:21.667711 (+ 1552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:11:21.667717 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:11:21.667958 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:21.668787 (+ 829us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:11:21.668815 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:11:21.671571 (+ 2756us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:11:21.676528 (+ 4957us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:11:21.676539 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:11:21.676556 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:11:21.677100 (+ 544us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:11:21.677877 (+ 777us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:11:21.677882 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:11:21.677885 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:11:21.678131 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:11:21.679337 (+ 1206us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:11:21.679344 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:11:21.679731 (+ 387us) client_negotiation.cc:769] Sending connection context
0811 09:11:21.679976 (+ 245us) client_negotiation.cc:240] Negotiation successful
0811 09:11:21.680642 (+ 666us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":599,"thread_start_us":383,"threads_started":1}
I20260811 09:11:21.682179 23972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:21.656527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.129:48911 (local address 127.28.154.190:36117)
0811 09:11:21.657171 (+ 644us) server_negotiation.cc:206] Beginning negotiation
0811 09:11:21.657187 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 09:11:21.657892 (+ 705us) server_negotiation.cc:407] Connection header received
0811 09:11:21.660114 (+ 2222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:11:21.660132 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:11:21.660653 (+ 521us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:11:21.661081 (+ 428us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:11:21.662758 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:21.664592 (+ 1834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:21.666328 (+ 1736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:21.667397 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:21.667657 (+ 260us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:21.671968 (+ 4311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:11:21.671980 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:11:21.672002 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:11:21.672061 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:11:21.676171 (+ 4110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:11:21.677281 (+ 1110us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:11:21.677292 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:11:21.677301 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:11:21.677563 (+ 262us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:11:21.678295 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:11:21.678301 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:11:21.678304 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:11:21.679133 (+ 829us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:11:21.679354 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0811 09:11:21.680306 (+ 952us) server_negotiation.cc:299] Negotiation successful
0811 09:11:21.680898 (+ 592us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":405,"thread_start_us":274,"threads_started":1}
I20260811 09:11:21.685901 23967 heartbeater.cc:347] Connected to a master server at 127.28.154.190:36117
I20260811 09:11:21.686326 23967 heartbeater.cc:464] Registering TS with master...
I20260811 09:11:21.687413 23967 heartbeater.cc:511] Master 127.28.154.190:36117 requested a full tablet report, sending...
I20260811 09:11:21.691686 23629 ts_manager.cc:194] Registered new tserver with Master: e04ee1cf144a4460abe550ff84ff1b29 (127.28.154.129:44993)
I20260811 09:11:21.693501 23629 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.154.129@KRBTEST.COM'} at 127.28.154.129:48911
W20260811 09:11:22.023058 23975 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:11:22.023363 23975 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:11:22.023496 23975 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:11:22.030908 23975 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:11:22.031039 23975 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:11:22.031189 23975 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.154.130
I20260811 09:11:22.041247 23975 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:32851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.130.keytab
--principal=kudu/127.28.154.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.154.130:41151
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.154.130
--webserver_port=34149
--tserver_master_addrs=127.28.154.190:36117
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.154.130
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:11:22.042661 23975 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:11:22.043972 23975 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:11:22.057381 23985 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:11:22.057412 23984 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:11:22.057871 23987 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:11:22.058367 23975 server_base.cc:1034] running on GCE node
I20260811 09:11:22.059018 23975 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:11:22.060114 23975 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:11:22.061388 23975 hybrid_clock.cc:648] HybridClock initialized: now 1786439482061324 us; error 48 us; skew 500 ppm
I20260811 09:11:22.061812 23975 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:11:22 dist-test-slave-xt4k krb5kdc[22586](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439482, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:11:22.065312 23975 init.cc:377] Logged in from keytab as kudu/127.28.154.130@KRBTEST.COM (short username kudu)
I20260811 09:11:22.066856 23975 webserver.cc:460] Webserver started at http://127.28.154.130:34149/ using document root <none> and password file <none>
I20260811 09:11:22.067734 23975 fs_manager.cc:359] Metadata directory not provided
I20260811 09:11:22.067883 23975 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:11:22.073864 23975 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.002s
I20260811 09:11:22.076938 24002 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:11:22.078410 23975 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 09:11:22.078583 23975 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-1/wal
uuid: "048c18609b164ee793f0e33a9ac111a7"
format_stamp: "Formatted at 2026-08-11 09:11:18 on dist-test-slave-xt4k"
server_key: "63e41ac7b26e1cd21bd24dd5da3362e8"
server_key_iv: "a4abc8b6a2eefce27f8af3025ab6be4d"
server_key_version: "encryptionkey@0"
I20260811 09:11:22.079666 23975 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:11:22.100605 23975 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:11:22.105962 23975 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:11:22.106289 23975 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:11:22.107739 23975 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:11:22.109899 23975 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:11:22.110029 23975 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:11:22.110212 23975 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:11:22.110292 23975 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:11:22.147670 23975 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.130:41151
I20260811 09:11:22.147723 24204 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.130:41151 every 8 connection(s)
I20260811 09:11:22.149331 23975 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-1/data/info.pb
I20260811 09:11:22.159471 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 23975
I20260811 09:11:22.160267 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.154.131:35775
--local_ip_for_outbound_sockets=127.28.154.131
--tserver_master_addrs=127.28.154.190:36117
--webserver_port=39891
--webserver_interface=127.28.154.131
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.131.keytab
--principal=kudu/127.28.154.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:32851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 09:11:22 dist-test-slave-xt4k krb5kdc[22586](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439482, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.130@KRBTEST.COM for kudu/127.28.154.190@KRBTEST.COM
I20260811 09:11:22.172036 23972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:22.152130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.130:36443 (local address 127.28.154.190:36117)
0811 09:11:22.152320 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 09:11:22.152324 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:11:22.153659 (+ 1335us) server_negotiation.cc:407] Connection header received
0811 09:11:22.155741 (+ 2082us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:11:22.155747 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:11:22.155862 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:11:22.155959 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:11:22.157610 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:22.158324 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:22.160360 (+ 2036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:22.161003 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:22.161153 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:22.165311 (+ 4158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:11:22.165316 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:11:22.165320 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:11:22.165385 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:11:22.168039 (+ 2654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:11:22.169139 (+ 1100us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:11:22.169144 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:11:22.169146 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:11:22.169306 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:11:22.169868 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:11:22.169873 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:11:22.169876 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:11:22.170392 (+ 516us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:11:22.170573 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0811 09:11:22.171419 (+ 846us) server_negotiation.cc:299] Negotiation successful
0811 09:11:22.171694 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 09:11:22.173092 24208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:22.152664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:36117 (local address 127.28.154.130:36443)
0811 09:11:22.153489 (+ 825us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:22.153532 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 09:11:22.155455 (+ 1923us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:11:22.156221 (+ 766us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:11:22.156229 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:11:22.156818 (+ 589us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:11:22.157423 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:11:22.157443 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:11:22.158504 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:11:22.158509 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:11:22.160100 (+ 1591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:11:22.160119 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:11:22.161287 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:11:22.161293 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:11:22.161566 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:22.162274 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:11:22.162300 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:11:22.164989 (+ 2689us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:11:22.168237 (+ 3248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:11:22.168250 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:11:22.168272 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:11:22.168953 (+ 681us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:11:22.169426 (+ 473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:11:22.169431 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:11:22.169434 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:11:22.169698 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:11:22.170583 (+ 885us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:11:22.170593 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:11:22.170991 (+ 398us) client_negotiation.cc:769] Sending connection context
0811 09:11:22.171255 (+ 264us) client_negotiation.cc:240] Negotiation successful
0811 09:11:22.171841 (+ 586us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":546,"thread_start_us":308,"threads_started":1}
I20260811 09:11:22.174818 24205 heartbeater.cc:347] Connected to a master server at 127.28.154.190:36117
I20260811 09:11:22.175148 24205 heartbeater.cc:464] Registering TS with master...
I20260811 09:11:22.175951 24205 heartbeater.cc:511] Master 127.28.154.190:36117 requested a full tablet report, sending...
I20260811 09:11:22.177631 23629 ts_manager.cc:194] Registered new tserver with Master: 048c18609b164ee793f0e33a9ac111a7 (127.28.154.130:41151)
I20260811 09:11:22.178678 23629 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.154.130@KRBTEST.COM'} at 127.28.154.130:36443
W20260811 09:11:22.514405 24209 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:11:22.514725 24209 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:11:22.514823 24209 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:11:22.522328 24209 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:11:22.522497 24209 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:11:22.522619 24209 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.154.131
I20260811 09:11:22.532797 24209 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:32851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.131.keytab
--principal=kudu/127.28.154.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.154.131:35775
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.154.131
--webserver_port=39891
--tserver_master_addrs=127.28.154.190:36117
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.154.131
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:11:22.534132 24209 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:11:22.535465 24209 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:11:22.550024 24221 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:11:22.549965 24227 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:11:22.550051 24222 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:11:22.550700 24209 server_base.cc:1034] running on GCE node
I20260811 09:11:22.551401 24209 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:11:22.552538 24209 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:11:22.553915 24209 hybrid_clock.cc:648] HybridClock initialized: now 1786439482553799 us; error 54 us; skew 500 ppm
I20260811 09:11:22.554581 24209 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:11:22 dist-test-slave-xt4k krb5kdc[22586](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439482, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:11:22.558086 24209 init.cc:377] Logged in from keytab as kudu/127.28.154.131@KRBTEST.COM (short username kudu)
I20260811 09:11:22.559819 24209 webserver.cc:460] Webserver started at http://127.28.154.131:39891/ using document root <none> and password file <none>
I20260811 09:11:22.560672 24209 fs_manager.cc:359] Metadata directory not provided
I20260811 09:11:22.560818 24209 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:11:22.566993 24209 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 09:11:22.570137 24236 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:11:22.571708 24209 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 09:11:22.571894 24209 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-2/wal
uuid: "699b1810d51a439684efddb537a4348e"
format_stamp: "Formatted at 2026-08-11 09:11:19 on dist-test-slave-xt4k"
server_key: "b375fe65da0c0f7e44fafa54c751e21c"
server_key_iv: "2fbd477725914d6faa1482a07ac32eb2"
server_key_version: "encryptionkey@0"
I20260811 09:11:22.573303 24209 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:11:22.592127 24209 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:11:22.596814 24209 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:11:22.597127 24209 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:11:22.598202 24209 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:11:22.599589 24209 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:11:22.599689 24209 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:11:22.599797 24209 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:11:22.599869 24209 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:11:22.633600 24209 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.131:35775
I20260811 09:11:22.633677 24450 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.131:35775 every 8 connection(s)
I20260811 09:11:22.635109 24209 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-2/data/info.pb
I20260811 09:11:22.642742 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 24209
Aug 11 09:11:22 dist-test-slave-xt4k krb5kdc[22586](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439482, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.131@KRBTEST.COM for kudu/127.28.154.190@KRBTEST.COM
I20260811 09:11:22.661537 23972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:22.637894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.131:48369 (local address 127.28.154.190:36117)
0811 09:11:22.638070 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 09:11:22.638075 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:11:22.639406 (+ 1331us) server_negotiation.cc:407] Connection header received
0811 09:11:22.641398 (+ 1992us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:11:22.641405 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:11:22.641488 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:11:22.641588 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:11:22.643617 (+ 2029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:22.644736 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:22.646355 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:22.646839 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:22.646964 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:22.654771 (+ 7807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:11:22.654779 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:11:22.654784 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:11:22.654857 (+ 73us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:11:22.658077 (+ 3220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:11:22.658969 (+ 892us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:11:22.658973 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:11:22.658980 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:11:22.659121 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:11:22.659659 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:11:22.659663 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:11:22.659665 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:11:22.659972 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:11:22.660117 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0811 09:11:22.660910 (+ 793us) server_negotiation.cc:299] Negotiation successful
0811 09:11:22.661155 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":57,"spinlock_wait_cycles":38528}
I20260811 09:11:22.661947 24462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:22.643915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58704 (local address 127.28.154.190:36117)
0811 09:11:22.644731 (+ 816us) server_negotiation.cc:206] Beginning negotiation
0811 09:11:22.644735 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:11:22.644751 (+ 16us) server_negotiation.cc:407] Connection header received
0811 09:11:22.644797 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:11:22.644800 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:11:22.644864 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:11:22.644965 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:11:22.646552 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:22.647629 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:22.649313 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:22.649778 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:22.649876 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:22.654771 (+ 4895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:11:22.654777 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:11:22.654781 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:11:22.654850 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:11:22.658517 (+ 3667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:11:22.659589 (+ 1072us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:11:22.659593 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:11:22.659595 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:11:22.659733 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:11:22.660166 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:11:22.660169 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:11:22.660171 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:11:22.660498 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:11:22.660740 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0811 09:11:22.661082 (+ 342us) server_negotiation.cc:299] Negotiation successful
0811 09:11:22.661315 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":722,"spinlock_wait_cycles":3968,"thread_start_us":558,"threads_started":1}
I20260811 09:11:22.662865 24455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:22.638441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:36117 (local address 127.28.154.131:48369)
0811 09:11:22.639219 (+ 778us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:22.639264 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 09:11:22.641080 (+ 1816us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:11:22.641853 (+ 773us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:11:22.641862 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:11:22.642556 (+ 694us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:11:22.643417 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:11:22.643445 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:11:22.644900 (+ 1455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:11:22.644904 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:11:22.646200 (+ 1296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:11:22.646214 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:11:22.647137 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:11:22.647145 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:11:22.647460 (+ 315us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:22.651761 (+ 4301us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:11:22.651788 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:11:22.654419 (+ 2631us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:11:22.658272 (+ 3853us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:11:22.658281 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:11:22.658295 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:11:22.658831 (+ 536us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:11:22.659252 (+ 421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:11:22.659259 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:11:22.659262 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:11:22.659537 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:11:22.660117 (+ 580us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:11:22.660125 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:11:22.660529 (+ 404us) client_negotiation.cc:769] Sending connection context
0811 09:11:22.660794 (+ 265us) client_negotiation.cc:240] Negotiation successful
0811 09:11:22.661368 (+ 574us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":516,"thread_start_us":315,"threads_started":1}
I20260811 09:11:22.664276 24453 heartbeater.cc:347] Connected to a master server at 127.28.154.190:36117
I20260811 09:11:22.664614 24453 heartbeater.cc:464] Registering TS with master...
I20260811 09:11:22.665355 24453 heartbeater.cc:511] Master 127.28.154.190:36117 requested a full tablet report, sending...
I20260811 09:11:22.666873 23628 ts_manager.cc:194] Registered new tserver with Master: 699b1810d51a439684efddb537a4348e (127.28.154.131:35775)
I20260811 09:11:22.667801 23628 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.154.131@KRBTEST.COM'} at 127.28.154.131:48369
I20260811 09:11:22.677686 29290 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 09:11:22.686304 24462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:22.678789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58720 (local address 127.28.154.190:36117)
0811 09:11:22.679000 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0811 09:11:22.679007 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:11:22.679572 (+ 565us) server_negotiation.cc:407] Connection header received
0811 09:11:22.679799 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:11:22.679804 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:11:22.679968 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:11:22.680114 (+ 146us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:11:22.680678 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:22.681459 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:22.683729 (+ 2270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:22.684251 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:22.684433 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:22.685061 (+ 628us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:11:22.685610 (+ 549us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:11:22.685740 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0811 09:11:22.685974 (+ 234us) server_negotiation.cc:299] Negotiation successful
0811 09:11:22.686067 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 09:11:22.687338 23628 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:58720:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 09:11:22.694823 23628 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 09:11:22.698027 23967 heartbeater.cc:503] Master 127.28.154.190:36117 was elected leader, sending a full tablet report...
I20260811 09:11:22.731524 24474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:22.708150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.129:44993 (local address 127.0.0.1:38134)
0811 09:11:22.721429 (+ 13279us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:22.721487 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 09:11:22.721669 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:11:22.723476 (+ 1807us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:11:22.723484 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:11:22.723516 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:11:22.723970 (+ 454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:11:22.723984 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:11:22.726216 (+ 2232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:11:22.726222 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:11:22.728412 (+ 2190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:11:22.728426 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:11:22.730349 (+ 1923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:11:22.730355 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:11:22.730564 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:22.730619 (+ 55us) client_negotiation.cc:769] Sending connection context
0811 09:11:22.730775 (+ 156us) client_negotiation.cc:240] Negotiation successful
0811 09:11:22.731006 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12988,"mutex_wait_us":33,"thread_start_us":14432,"threads_started":1}
I20260811 09:11:22.731626 24473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:22.708850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40796 (local address 127.28.154.131:35775)
0811 09:11:22.722394 (+ 13544us) server_negotiation.cc:206] Beginning negotiation
0811 09:11:22.722405 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 09:11:22.723004 (+ 599us) server_negotiation.cc:407] Connection header received
0811 09:11:22.723182 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:11:22.723190 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:11:22.723446 (+ 256us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:11:22.723645 (+ 199us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:11:22.724316 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:22.725959 (+ 1643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:22.728789 (+ 2830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:22.730149 (+ 1360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:22.730324 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:22.730516 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0811 09:11:22.731127 (+ 611us) server_negotiation.cc:299] Negotiation successful
0811 09:11:22.731319 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13342,"thread_start_us":13188,"threads_started":1}
I20260811 09:11:22.732451 24468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:22.709057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.131:35775 (local address 127.0.0.1:40796)
0811 09:11:22.722858 (+ 13801us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:22.722879 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 09:11:22.723012 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:11:22.723714 (+ 702us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:11:22.723718 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:11:22.723740 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:11:22.724130 (+ 390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:11:22.724141 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:11:22.726143 (+ 2002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:11:22.726149 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:11:22.728618 (+ 2469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:11:22.728635 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:11:22.730708 (+ 2073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:11:22.730717 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:11:22.730914 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:22.730948 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 09:11:22.731896 (+ 948us) client_negotiation.cc:240] Negotiation successful
0811 09:11:22.732047 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13673,"mutex_wait_us":83,"thread_start_us":11895,"threads_started":1}
I20260811 09:11:22.732635 24470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:22.708236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38134 (local address 127.28.154.129:44993)
0811 09:11:22.722803 (+ 14567us) server_negotiation.cc:206] Beginning negotiation
0811 09:11:22.722811 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:11:22.722842 (+ 31us) server_negotiation.cc:407] Connection header received
0811 09:11:22.722975 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:11:22.722985 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:11:22.723245 (+ 260us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:11:22.723426 (+ 181us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:11:22.724240 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:22.725946 (+ 1706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:22.728581 (+ 2635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:22.730102 (+ 1521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:22.730324 (+ 222us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:22.730516 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0811 09:11:22.732076 (+ 1560us) server_negotiation.cc:299] Negotiation successful
0811 09:11:22.732257 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14395,"thread_start_us":14230,"threads_started":1}
I20260811 09:11:22.733853 24479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:22.724358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.130:41151 (local address 127.0.0.1:57024)
0811 09:11:22.726192 (+ 1834us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:22.726218 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 09:11:22.726364 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:11:22.727023 (+ 659us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:11:22.727029 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:11:22.727056 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:11:22.727575 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:11:22.727594 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:11:22.729277 (+ 1683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:11:22.729282 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:11:22.730806 (+ 1524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:11:22.730819 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:11:22.732991 (+ 2172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:11:22.732998 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:11:22.733170 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:22.733202 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 09:11:22.733297 (+ 95us) client_negotiation.cc:240] Negotiation successful
0811 09:11:22.733538 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1689,"thread_start_us":1548,"threads_started":1}
I20260811 09:11:22.734223 24478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:22.723364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57024 (local address 127.28.154.130:41151)
0811 09:11:22.724260 (+ 896us) server_negotiation.cc:206] Beginning negotiation
0811 09:11:22.724267 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:11:22.726340 (+ 2073us) server_negotiation.cc:407] Connection header received
0811 09:11:22.726547 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:11:22.726555 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:11:22.726808 (+ 253us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:11:22.727022 (+ 214us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:11:22.727783 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:22.729146 (+ 1363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:22.730990 (+ 1844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:22.732468 (+ 1478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:22.732602 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:22.732778 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0811 09:11:22.733386 (+ 608us) server_negotiation.cc:299] Negotiation successful
0811 09:11:22.733700 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":762,"thread_start_us":647,"threads_started":1}
I20260811 09:11:22.737068 24337 tablet_service.cc:1467] Processing CreateTablet for tablet 657857d753744a92a6087437684a0e64 (DEFAULT_TABLE table=test-table [id=13bc68e0b1544fb7813ff90469da4f3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:11:22.738679 23847 tablet_service.cc:1467] Processing CreateTablet for tablet 657857d753744a92a6087437684a0e64 (DEFAULT_TABLE table=test-table [id=13bc68e0b1544fb7813ff90469da4f3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:11:22.739886 24337 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 657857d753744a92a6087437684a0e64. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:11:22.740425 23847 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 657857d753744a92a6087437684a0e64. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:11:22.741884 24094 tablet_service.cc:1467] Processing CreateTablet for tablet 657857d753744a92a6087437684a0e64 (DEFAULT_TABLE table=test-table [id=13bc68e0b1544fb7813ff90469da4f3f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:11:22.743362 24094 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 657857d753744a92a6087437684a0e64. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:11:22.753466 24480 tablet_bootstrap.cc:492] T 657857d753744a92a6087437684a0e64 P e04ee1cf144a4460abe550ff84ff1b29: Bootstrap starting.
I20260811 09:11:22.757072 24480 tablet_bootstrap.cc:654] T 657857d753744a92a6087437684a0e64 P e04ee1cf144a4460abe550ff84ff1b29: Neither blocks nor log segments found. Creating new log.
I20260811 09:11:22.758234 24482 tablet_bootstrap.cc:492] T 657857d753744a92a6087437684a0e64 P 699b1810d51a439684efddb537a4348e: Bootstrap starting.
I20260811 09:11:22.758639 24480 log.cc:824] T 657857d753744a92a6087437684a0e64 P e04ee1cf144a4460abe550ff84ff1b29: Log is configured to *not* fsync() on all Append() calls
I20260811 09:11:22.762656 24482 tablet_bootstrap.cc:654] T 657857d753744a92a6087437684a0e64 P 699b1810d51a439684efddb537a4348e: Neither blocks nor log segments found. Creating new log.
I20260811 09:11:22.763805 24485 tablet_bootstrap.cc:492] T 657857d753744a92a6087437684a0e64 P 048c18609b164ee793f0e33a9ac111a7: Bootstrap starting.
I20260811 09:11:22.765044 24482 log.cc:824] T 657857d753744a92a6087437684a0e64 P 699b1810d51a439684efddb537a4348e: Log is configured to *not* fsync() on all Append() calls
I20260811 09:11:22.767513 24480 tablet_bootstrap.cc:492] T 657857d753744a92a6087437684a0e64 P e04ee1cf144a4460abe550ff84ff1b29: No bootstrap required, opened a new log
I20260811 09:11:22.767859 24480 ts_tablet_manager.cc:1397] T 657857d753744a92a6087437684a0e64 P e04ee1cf144a4460abe550ff84ff1b29: Time spent bootstrapping tablet: real 0.015s user 0.010s sys 0.000s
I20260811 09:11:22.769639 24482 tablet_bootstrap.cc:492] T 657857d753744a92a6087437684a0e64 P 699b1810d51a439684efddb537a4348e: No bootstrap required, opened a new log
I20260811 09:11:22.769958 24482 ts_tablet_manager.cc:1397] T 657857d753744a92a6087437684a0e64 P 699b1810d51a439684efddb537a4348e: Time spent bootstrapping tablet: real 0.012s user 0.010s sys 0.000s
I20260811 09:11:22.770042 24485 tablet_bootstrap.cc:654] T 657857d753744a92a6087437684a0e64 P 048c18609b164ee793f0e33a9ac111a7: Neither blocks nor log segments found. Creating new log.
I20260811 09:11:22.771118 24485 log.cc:824] T 657857d753744a92a6087437684a0e64 P 048c18609b164ee793f0e33a9ac111a7: Log is configured to *not* fsync() on all Append() calls
I20260811 09:11:22.774608 24485 tablet_bootstrap.cc:492] T 657857d753744a92a6087437684a0e64 P 048c18609b164ee793f0e33a9ac111a7: No bootstrap required, opened a new log
I20260811 09:11:22.774897 24485 ts_tablet_manager.cc:1397] T 657857d753744a92a6087437684a0e64 P 048c18609b164ee793f0e33a9ac111a7: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.000s
I20260811 09:11:22.779790 24480 raft_consensus.cc:348] T 657857d753744a92a6087437684a0e64 P e04ee1cf144a4460abe550ff84ff1b29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e04ee1cf144a4460abe550ff84ff1b29" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 44993 } } peers { permanent_uuid: "048c18609b164ee793f0e33a9ac111a7" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 41151 } } peers { permanent_uuid: "699b1810d51a439684efddb537a4348e" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 35775 } }
I20260811 09:11:22.780369 24480 raft_consensus.cc:374] T 657857d753744a92a6087437684a0e64 P e04ee1cf144a4460abe550ff84ff1b29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:11:22.780480 24480 raft_consensus.cc:729] T 657857d753744a92a6087437684a0e64 P e04ee1cf144a4460abe550ff84ff1b29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e04ee1cf144a4460abe550ff84ff1b29, State: Initialized, Role: FOLLOWER
I20260811 09:11:22.781013 24482 raft_consensus.cc:348] T 657857d753744a92a6087437684a0e64 P 699b1810d51a439684efddb537a4348e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e04ee1cf144a4460abe550ff84ff1b29" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 44993 } } peers { permanent_uuid: "048c18609b164ee793f0e33a9ac111a7" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 41151 } } peers { permanent_uuid: "699b1810d51a439684efddb537a4348e" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 35775 } }
I20260811 09:11:22.781342 24480 consensus_queue.cc:260] T 657857d753744a92a6087437684a0e64 P e04ee1cf144a4460abe550ff84ff1b29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e04ee1cf144a4460abe550ff84ff1b29" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 44993 } } peers { permanent_uuid: "048c18609b164ee793f0e33a9ac111a7" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 41151 } } peers { permanent_uuid: "699b1810d51a439684efddb537a4348e" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 35775 } }
I20260811 09:11:22.781591 24482 raft_consensus.cc:374] T 657857d753744a92a6087437684a0e64 P 699b1810d51a439684efddb537a4348e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:11:22.781682 24482 raft_consensus.cc:729] T 657857d753744a92a6087437684a0e64 P 699b1810d51a439684efddb537a4348e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 699b1810d51a439684efddb537a4348e, State: Initialized, Role: FOLLOWER
I20260811 09:11:22.782606 24482 consensus_queue.cc:260] T 657857d753744a92a6087437684a0e64 P 699b1810d51a439684efddb537a4348e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e04ee1cf144a4460abe550ff84ff1b29" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 44993 } } peers { permanent_uuid: "048c18609b164ee793f0e33a9ac111a7" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 41151 } } peers { permanent_uuid: "699b1810d51a439684efddb537a4348e" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 35775 } }
I20260811 09:11:22.783591 24480 ts_tablet_manager.cc:1428] T 657857d753744a92a6087437684a0e64 P e04ee1cf144a4460abe550ff84ff1b29: Time spent starting tablet: real 0.016s user 0.018s sys 0.000s
I20260811 09:11:22.784235 24485 raft_consensus.cc:348] T 657857d753744a92a6087437684a0e64 P 048c18609b164ee793f0e33a9ac111a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e04ee1cf144a4460abe550ff84ff1b29" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 44993 } } peers { permanent_uuid: "048c18609b164ee793f0e33a9ac111a7" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 41151 } } peers { permanent_uuid: "699b1810d51a439684efddb537a4348e" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 35775 } }
I20260811 09:11:22.784775 24485 raft_consensus.cc:374] T 657857d753744a92a6087437684a0e64 P 048c18609b164ee793f0e33a9ac111a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:11:22.784893 24485 raft_consensus.cc:729] T 657857d753744a92a6087437684a0e64 P 048c18609b164ee793f0e33a9ac111a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 048c18609b164ee793f0e33a9ac111a7, State: Initialized, Role: FOLLOWER
I20260811 09:11:22.785681 24482 ts_tablet_manager.cc:1428] T 657857d753744a92a6087437684a0e64 P 699b1810d51a439684efddb537a4348e: Time spent starting tablet: real 0.016s user 0.016s sys 0.000s
I20260811 09:11:22.785506 24485 consensus_queue.cc:260] T 657857d753744a92a6087437684a0e64 P 048c18609b164ee793f0e33a9ac111a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e04ee1cf144a4460abe550ff84ff1b29" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 44993 } } peers { permanent_uuid: "048c18609b164ee793f0e33a9ac111a7" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 41151 } } peers { permanent_uuid: "699b1810d51a439684efddb537a4348e" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 35775 } }
I20260811 09:11:22.786135 24453 heartbeater.cc:503] Master 127.28.154.190:36117 was elected leader, sending a full tablet report...
I20260811 09:11:22.787014 24205 heartbeater.cc:503] Master 127.28.154.190:36117 was elected leader, sending a full tablet report...
I20260811 09:11:22.790887 24485 ts_tablet_manager.cc:1428] T 657857d753744a92a6087437684a0e64 P 048c18609b164ee793f0e33a9ac111a7: Time spent starting tablet: real 0.016s user 0.010s sys 0.005s
W20260811 09:11:22.886989 24454 tablet.cc:2369] T 657857d753744a92a6087437684a0e64 P 699b1810d51a439684efddb537a4348e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 09:11:22.901499 24207 tablet.cc:2369] T 657857d753744a92a6087437684a0e64 P 048c18609b164ee793f0e33a9ac111a7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 09:11:22.906036 23970 tablet.cc:2369] T 657857d753744a92a6087437684a0e64 P e04ee1cf144a4460abe550ff84ff1b29: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 09:11:23.061245 24490 raft_consensus.cc:482] T 657857d753744a92a6087437684a0e64 P e04ee1cf144a4460abe550ff84ff1b29 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:11:23.061578 24490 raft_consensus.cc:504] T 657857d753744a92a6087437684a0e64 P e04ee1cf144a4460abe550ff84ff1b29 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e04ee1cf144a4460abe550ff84ff1b29" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 44993 } } peers { permanent_uuid: "048c18609b164ee793f0e33a9ac111a7" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 41151 } } peers { permanent_uuid: "699b1810d51a439684efddb537a4348e" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 35775 } }
I20260811 09:11:23.063805 24490 leader_election.cc:288] T 657857d753744a92a6087437684a0e64 P e04ee1cf144a4460abe550ff84ff1b29 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 048c18609b164ee793f0e33a9ac111a7 (127.28.154.130:41151), 699b1810d51a439684efddb537a4348e (127.28.154.131:35775)
I20260811 09:11:23.071324 24478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:23.064305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.129:33161 (local address 127.28.154.130:41151)
0811 09:11:23.064608 (+ 303us) server_negotiation.cc:206] Beginning negotiation
0811 09:11:23.064615 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:11:23.065780 (+ 1165us) server_negotiation.cc:407] Connection header received
0811 09:11:23.065957 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:11:23.065961 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:11:23.066022 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:11:23.066096 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:11:23.066800 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:23.067540 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:23.069571 (+ 2031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:23.070329 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:23.070477 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:23.070559 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 09:11:23.070946 (+ 387us) server_negotiation.cc:299] Negotiation successful
0811 09:11:23.071079 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
I20260811 09:11:23.071324 24473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:23.064427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.129:46249 (local address 127.28.154.131:35775)
0811 09:11:23.064615 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 09:11:23.064621 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:11:23.065754 (+ 1133us) server_negotiation.cc:407] Connection header received
0811 09:11:23.065938 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:11:23.065943 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:11:23.066009 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:11:23.066088 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:11:23.066798 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:23.067579 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:23.069563 (+ 1984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:23.070321 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:23.070486 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:23.070560 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 09:11:23.070920 (+ 360us) server_negotiation.cc:299] Negotiation successful
0811 09:11:23.071063 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 09:11:23.071501 24498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:23.064122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.131:35775 (local address 127.28.154.129:46249)
0811 09:11:23.065567 (+ 1445us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:23.065589 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 09:11:23.065779 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:11:23.066122 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:11:23.066126 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:11:23.066152 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:11:23.066643 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:11:23.066653 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:11:23.067714 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:11:23.067718 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:11:23.069389 (+ 1671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:11:23.069405 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:11:23.070565 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:11:23.070571 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:11:23.070742 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:23.070785 (+ 43us) client_negotiation.cc:769] Sending connection context
0811 09:11:23.070869 (+ 84us) client_negotiation.cc:240] Negotiation successful
0811 09:11:23.071065 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1169,"mutex_wait_us":118,"thread_start_us":529,"threads_started":1}
I20260811 09:11:23.071851 24499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:23.064377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.130:41151 (local address 127.28.154.129:33161)
0811 09:11:23.065652 (+ 1275us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:23.065668 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 09:11:23.065809 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:11:23.066133 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:11:23.066137 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:11:23.066181 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:11:23.066630 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:11:23.066641 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:11:23.067686 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:11:23.067690 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:11:23.069411 (+ 1721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:11:23.069426 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:11:23.070612 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:11:23.070617 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:11:23.070787 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:23.070811 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 09:11:23.070889 (+ 78us) client_negotiation.cc:240] Negotiation successful
0811 09:11:23.071031 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1154,"mutex_wait_us":45,"spinlock_wait_cycles":2432,"thread_start_us":898,"threads_started":1}
I20260811 09:11:23.072634 24374 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "657857d753744a92a6087437684a0e64" candidate_uuid: "e04ee1cf144a4460abe550ff84ff1b29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "699b1810d51a439684efddb537a4348e" is_pre_election: true
I20260811 09:11:23.072785 24123 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "657857d753744a92a6087437684a0e64" candidate_uuid: "e04ee1cf144a4460abe550ff84ff1b29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "048c18609b164ee793f0e33a9ac111a7" is_pre_election: true
I20260811 09:11:23.073158 24374 raft_consensus.cc:2454] T 657857d753744a92a6087437684a0e64 P 699b1810d51a439684efddb537a4348e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e04ee1cf144a4460abe550ff84ff1b29 in term 0.
I20260811 09:11:23.073498 24123 raft_consensus.cc:2454] T 657857d753744a92a6087437684a0e64 P 048c18609b164ee793f0e33a9ac111a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e04ee1cf144a4460abe550ff84ff1b29 in term 0.
I20260811 09:11:23.073915 23766 leader_election.cc:302] T 657857d753744a92a6087437684a0e64 P e04ee1cf144a4460abe550ff84ff1b29 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 699b1810d51a439684efddb537a4348e, e04ee1cf144a4460abe550ff84ff1b29; no voters:
I20260811 09:11:23.074376 24490 raft_consensus.cc:2781] T 657857d753744a92a6087437684a0e64 P e04ee1cf144a4460abe550ff84ff1b29 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 09:11:23.074555 24490 raft_consensus.cc:482] T 657857d753744a92a6087437684a0e64 P e04ee1cf144a4460abe550ff84ff1b29 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 09:11:23.074667 24490 raft_consensus.cc:3037] T 657857d753744a92a6087437684a0e64 P e04ee1cf144a4460abe550ff84ff1b29 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:11:23.077250 24490 raft_consensus.cc:504] T 657857d753744a92a6087437684a0e64 P e04ee1cf144a4460abe550ff84ff1b29 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e04ee1cf144a4460abe550ff84ff1b29" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 44993 } } peers { permanent_uuid: "048c18609b164ee793f0e33a9ac111a7" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 41151 } } peers { permanent_uuid: "699b1810d51a439684efddb537a4348e" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 35775 } }
I20260811 09:11:23.078420 24490 leader_election.cc:288] T 657857d753744a92a6087437684a0e64 P e04ee1cf144a4460abe550ff84ff1b29 [CANDIDATE]: Term 1 election: Requested vote from peers 048c18609b164ee793f0e33a9ac111a7 (127.28.154.130:41151), 699b1810d51a439684efddb537a4348e (127.28.154.131:35775)
I20260811 09:11:23.079166 24123 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "657857d753744a92a6087437684a0e64" candidate_uuid: "e04ee1cf144a4460abe550ff84ff1b29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "048c18609b164ee793f0e33a9ac111a7"
I20260811 09:11:23.079306 24374 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "657857d753744a92a6087437684a0e64" candidate_uuid: "e04ee1cf144a4460abe550ff84ff1b29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "699b1810d51a439684efddb537a4348e"
I20260811 09:11:23.079396 24123 raft_consensus.cc:3037] T 657857d753744a92a6087437684a0e64 P 048c18609b164ee793f0e33a9ac111a7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:11:23.079545 24374 raft_consensus.cc:3037] T 657857d753744a92a6087437684a0e64 P 699b1810d51a439684efddb537a4348e [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:11:23.082340 24374 raft_consensus.cc:2454] T 657857d753744a92a6087437684a0e64 P 699b1810d51a439684efddb537a4348e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e04ee1cf144a4460abe550ff84ff1b29 in term 1.
I20260811 09:11:23.082885 23766 leader_election.cc:302] T 657857d753744a92a6087437684a0e64 P e04ee1cf144a4460abe550ff84ff1b29 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 699b1810d51a439684efddb537a4348e, e04ee1cf144a4460abe550ff84ff1b29; no voters:
I20260811 09:11:23.082973 24123 raft_consensus.cc:2454] T 657857d753744a92a6087437684a0e64 P 048c18609b164ee793f0e33a9ac111a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e04ee1cf144a4460abe550ff84ff1b29 in term 1.
I20260811 09:11:23.083192 24490 raft_consensus.cc:2781] T 657857d753744a92a6087437684a0e64 P e04ee1cf144a4460abe550ff84ff1b29 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:11:23.084026 24490 raft_consensus.cc:686] T 657857d753744a92a6087437684a0e64 P e04ee1cf144a4460abe550ff84ff1b29 [term 1 LEADER]: Becoming Leader. State: Replica: e04ee1cf144a4460abe550ff84ff1b29, State: Running, Role: LEADER
I20260811 09:11:23.084769 24490 consensus_queue.cc:237] T 657857d753744a92a6087437684a0e64 P e04ee1cf144a4460abe550ff84ff1b29 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e04ee1cf144a4460abe550ff84ff1b29" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 44993 } } peers { permanent_uuid: "048c18609b164ee793f0e33a9ac111a7" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 41151 } } peers { permanent_uuid: "699b1810d51a439684efddb537a4348e" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 35775 } }
I20260811 09:11:23.090524 23627 catalog_manager.cc:5495] T 657857d753744a92a6087437684a0e64 P e04ee1cf144a4460abe550ff84ff1b29 reported cstate change: term changed from 0 to 1, leader changed from <none> to e04ee1cf144a4460abe550ff84ff1b29 (127.28.154.129). New cstate: current_term: 1 leader_uuid: "e04ee1cf144a4460abe550ff84ff1b29" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e04ee1cf144a4460abe550ff84ff1b29" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 44993 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "048c18609b164ee793f0e33a9ac111a7" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 41151 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "699b1810d51a439684efddb537a4348e" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 35775 } health_report { overall_health: UNKNOWN } } }
I20260811 09:11:23.185153 24470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:23.178636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38148 (local address 127.28.154.129:44993)
0811 09:11:23.179071 (+ 435us) server_negotiation.cc:206] Beginning negotiation
0811 09:11:23.179076 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:11:23.179092 (+ 16us) server_negotiation.cc:407] Connection header received
0811 09:11:23.179185 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:11:23.179188 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:11:23.179270 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:11:23.179414 (+ 144us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:11:23.179833 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:23.180633 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:23.182193 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:23.182700 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:23.182819 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:23.183389 (+ 570us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:11:23.184112 (+ 723us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:11:23.184393 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0811 09:11:23.184667 (+ 274us) server_negotiation.cc:299] Negotiation successful
0811 09:11:23.184837 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":263}
I20260811 09:11:23.194756 24123 raft_consensus.cc:1261] T 657857d753744a92a6087437684a0e64 P 048c18609b164ee793f0e33a9ac111a7 [term 1 FOLLOWER]: Refusing update from remote peer e04ee1cf144a4460abe550ff84ff1b29: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:11:23.194756 24374 raft_consensus.cc:1261] T 657857d753744a92a6087437684a0e64 P 699b1810d51a439684efddb537a4348e [term 1 FOLLOWER]: Refusing update from remote peer e04ee1cf144a4460abe550ff84ff1b29: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:11:23.196390 24500 consensus_queue.cc:1035] T 657857d753744a92a6087437684a0e64 P e04ee1cf144a4460abe550ff84ff1b29 [LEADER]: Connected to new peer: Peer: permanent_uuid: "048c18609b164ee793f0e33a9ac111a7" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 41151 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:11:23.196877 24490 consensus_queue.cc:1035] T 657857d753744a92a6087437684a0e64 P e04ee1cf144a4460abe550ff84ff1b29 [LEADER]: Connected to new peer: Peer: permanent_uuid: "699b1810d51a439684efddb537a4348e" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 35775 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:11:23.223366 23627 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:58720:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:11:23.223721 23627 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:58720:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:11:23.227733 23627 catalog_manager.cc:5782] T 00000000000000000000000000000000 P adc54f73b9dc4ef09d9ede64155cd1e5: Sending DeleteTablet for 3 replicas of tablet 657857d753744a92a6087437684a0e64
I20260811 09:11:23.229179 24094 tablet_service.cc:1514] Processing DeleteTablet for tablet 657857d753744a92a6087437684a0e64 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:11:23 UTC) from {username='kudu', principal='kudu/127.28.154.190@KRBTEST.COM'} at 127.0.0.1:57024
I20260811 09:11:23.229185 23847 tablet_service.cc:1514] Processing DeleteTablet for tablet 657857d753744a92a6087437684a0e64 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:11:23 UTC) from {username='kudu', principal='kudu/127.28.154.190@KRBTEST.COM'} at 127.0.0.1:38134
I20260811 09:11:23.229172 24337 tablet_service.cc:1514] Processing DeleteTablet for tablet 657857d753744a92a6087437684a0e64 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:11:23 UTC) from {username='kudu', principal='kudu/127.28.154.190@KRBTEST.COM'} at 127.0.0.1:40796
I20260811 09:11:23.230701 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 23700
I20260811 09:11:23.244874 24527 tablet_replica.cc:330] T 657857d753744a92a6087437684a0e64 P 048c18609b164ee793f0e33a9ac111a7: stopping tablet replica
I20260811 09:11:23.245056 24528 tablet_replica.cc:330] T 657857d753744a92a6087437684a0e64 P 699b1810d51a439684efddb537a4348e: stopping tablet replica
I20260811 09:11:23.245510 24528 raft_consensus.cc:2229] T 657857d753744a92a6087437684a0e64 P 699b1810d51a439684efddb537a4348e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:11:23.245481 24527 raft_consensus.cc:2229] T 657857d753744a92a6087437684a0e64 P 048c18609b164ee793f0e33a9ac111a7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:11:23.246287 24528 raft_consensus.cc:2258] T 657857d753744a92a6087437684a0e64 P 699b1810d51a439684efddb537a4348e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:11:23.246354 24527 raft_consensus.cc:2258] T 657857d753744a92a6087437684a0e64 P 048c18609b164ee793f0e33a9ac111a7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:11:23.249121 24528 ts_tablet_manager.cc:1905] T 657857d753744a92a6087437684a0e64 P 699b1810d51a439684efddb537a4348e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:11:23.249157 24527 ts_tablet_manager.cc:1905] T 657857d753744a92a6087437684a0e64 P 048c18609b164ee793f0e33a9ac111a7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:11:23.257954 24528 ts_tablet_manager.cc:1918] T 657857d753744a92a6087437684a0e64 P 699b1810d51a439684efddb537a4348e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:11:23.258167 24528 log.cc:1196] T 657857d753744a92a6087437684a0e64 P 699b1810d51a439684efddb537a4348e: Deleting WAL directory at /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-2/wal/wals/657857d753744a92a6087437684a0e64
I20260811 09:11:23.258303 24527 ts_tablet_manager.cc:1918] T 657857d753744a92a6087437684a0e64 P 048c18609b164ee793f0e33a9ac111a7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:11:23.258500 24527 log.cc:1196] T 657857d753744a92a6087437684a0e64 P 048c18609b164ee793f0e33a9ac111a7: Deleting WAL directory at /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-1/wal/wals/657857d753744a92a6087437684a0e64
I20260811 09:11:23.258780 24528 ts_tablet_manager.cc:1939] T 657857d753744a92a6087437684a0e64 P 699b1810d51a439684efddb537a4348e: Deleting consensus metadata
I20260811 09:11:23.259255 24527 ts_tablet_manager.cc:1939] T 657857d753744a92a6087437684a0e64 P 048c18609b164ee793f0e33a9ac111a7: Deleting consensus metadata
I20260811 09:11:23.260923 23609 catalog_manager.cc:4841] TS 699b1810d51a439684efddb537a4348e (127.28.154.131:35775): tablet 657857d753744a92a6087437684a0e64 (table test-table [id=13bc68e0b1544fb7813ff90469da4f3f]) successfully deleted
I20260811 09:11:23.261178 24526 tablet_replica.cc:330] T 657857d753744a92a6087437684a0e64 P e04ee1cf144a4460abe550ff84ff1b29: stopping tablet replica
I20260811 09:11:23.261324 23605 catalog_manager.cc:4841] TS 048c18609b164ee793f0e33a9ac111a7 (127.28.154.130:41151): tablet 657857d753744a92a6087437684a0e64 (table test-table [id=13bc68e0b1544fb7813ff90469da4f3f]) successfully deleted
I20260811 09:11:23.261798 24526 raft_consensus.cc:2229] T 657857d753744a92a6087437684a0e64 P e04ee1cf144a4460abe550ff84ff1b29 [term 1 LEADER]: Raft consensus shutting down.
I20260811 09:11:23.262529 24526 raft_consensus.cc:2258] T 657857d753744a92a6087437684a0e64 P e04ee1cf144a4460abe550ff84ff1b29 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:11:23.265197 24526 ts_tablet_manager.cc:1905] T 657857d753744a92a6087437684a0e64 P e04ee1cf144a4460abe550ff84ff1b29: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:11:23.272876 24526 ts_tablet_manager.cc:1918] T 657857d753744a92a6087437684a0e64 P e04ee1cf144a4460abe550ff84ff1b29: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:11:23.273247 24526 log.cc:1196] T 657857d753744a92a6087437684a0e64 P e04ee1cf144a4460abe550ff84ff1b29: Deleting WAL directory at /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-0/wal/wals/657857d753744a92a6087437684a0e64
I20260811 09:11:23.273864 24526 ts_tablet_manager.cc:1939] T 657857d753744a92a6087437684a0e64 P e04ee1cf144a4460abe550ff84ff1b29: Deleting consensus metadata
I20260811 09:11:23.276088 23605 catalog_manager.cc:4841] TS e04ee1cf144a4460abe550ff84ff1b29 (127.28.154.129:44993): tablet 657857d753744a92a6087437684a0e64 (table test-table [id=13bc68e0b1544fb7813ff90469da4f3f]) successfully deleted
I20260811 09:11:23.279322 24470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:23.231942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38154 (local address 127.28.154.129:44993)
0811 09:11:23.232138 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 09:11:23.232145 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:11:23.256437 (+ 24292us) server_negotiation.cc:407] Connection header received
0811 09:11:23.256998 (+ 561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:11:23.257004 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:11:23.257093 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:11:23.257162 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:11:23.260297 (+ 3135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:23.261499 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:23.263575 (+ 2076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:23.264268 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:23.264467 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:23.268562 (+ 4095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:11:23.268583 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:11:23.268605 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:11:23.268674 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:11:23.274658 (+ 5984us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:11:23.275878 (+ 1220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:11:23.275924 (+ 46us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:11:23.275927 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:11:23.276318 (+ 391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:11:23.277423 (+ 1105us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:11:23.277427 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:11:23.277430 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:11:23.277940 (+ 510us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:11:23.278258 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0811 09:11:23.278645 (+ 387us) server_negotiation.cc:299] Negotiation successful
0811 09:11:23.278900 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 09:11:23.280105 23960 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:11:23.405429 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 23700
I20260811 09:11:23.435832 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 23975
I20260811 09:11:23.455178 24478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:23.439377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57040 (local address 127.28.154.130:41151)
0811 09:11:23.439619 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0811 09:11:23.439624 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:11:23.439646 (+ 22us) server_negotiation.cc:407] Connection header received
0811 09:11:23.439710 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:11:23.439714 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:11:23.439786 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:11:23.439876 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:11:23.441448 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:23.442185 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:23.444129 (+ 1944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:23.444645 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:23.444754 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:23.447930 (+ 3176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:11:23.447948 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:11:23.447967 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:11:23.448012 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:11:23.451312 (+ 3300us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:11:23.452295 (+ 983us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:11:23.452304 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:11:23.452306 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:11:23.452630 (+ 324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:11:23.453160 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:11:23.453164 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:11:23.453166 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:11:23.453660 (+ 494us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:11:23.453991 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0811 09:11:23.454532 (+ 541us) server_negotiation.cc:299] Negotiation successful
0811 09:11:23.454795 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
I20260811 09:11:23.456210 24198 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:11:23.548405 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 23975
I20260811 09:11:23.574751 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 24209
I20260811 09:11:23.589764 24536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:23.575537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40810 (local address 127.28.154.131:35775)
0811 09:11:23.576079 (+ 542us) server_negotiation.cc:206] Beginning negotiation
0811 09:11:23.576084 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:11:23.576103 (+ 19us) server_negotiation.cc:407] Connection header received
0811 09:11:23.576161 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:11:23.576164 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:11:23.576236 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:11:23.576331 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:11:23.578026 (+ 1695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:23.578905 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:23.580496 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:23.580931 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:23.581039 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:23.583768 (+ 2729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:11:23.583784 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:11:23.583804 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:11:23.583843 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:11:23.586798 (+ 2955us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:11:23.587582 (+ 784us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:11:23.587591 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:11:23.587594 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:11:23.587752 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:11:23.588137 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:11:23.588140 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:11:23.588142 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:11:23.588589 (+ 447us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:11:23.588822 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0811 09:11:23.589197 (+ 375us) server_negotiation.cc:299] Negotiation successful
0811 09:11:23.589395 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":402,"thread_start_us":281,"threads_started":1}
I20260811 09:11:23.590581 24445 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:11:23.683441 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 24209
I20260811 09:11:23.729624 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 23572
I20260811 09:11:23.730856 23682 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:11:23.847515 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 23572
I20260811 09:11:23.879367 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.154.190:36117
--webserver_interface=127.28.154.190
--webserver_port=37189
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab
--principal=kudu/127.28.154.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:32851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.154.190:36117
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:11:24.242383 24540 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:11:24.242998 24540 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:11:24.243191 24540 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:11:24.251538 24540 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:11:24.251713 24540 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:11:24.251775 24540 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:11:24.251860 24540 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:11:24.252035 24540 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:11:24.264118 24540 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:32851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.154.190:36117
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab
--principal=kudu/127.28.154.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.154.190:36117
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.154.190
--webserver_port=37189
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:11:24.265604 24540 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:11:24.266983 24540 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:11:24.279529 24550 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:11:24.279573 24551 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:11:24.279609 24556 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:11:24.280426 24540 server_base.cc:1034] running on GCE node
I20260811 09:11:24.281287 24540 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:11:24.282920 24540 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:11:24.284170 24540 hybrid_clock.cc:648] HybridClock initialized: now 1786439484284109 us; error 49 us; skew 500 ppm
I20260811 09:11:24.284591 24540 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:11:24 dist-test-slave-xt4k krb5kdc[22586](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439484, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:11:24.288034 24540 init.cc:377] Logged in from keytab as kudu/127.28.154.190@KRBTEST.COM (short username kudu)
I20260811 09:11:24.289779 24540 webserver.cc:460] Webserver started at http://127.28.154.190:37189/ using document root <none> and password file <none>
I20260811 09:11:24.290514 24540 fs_manager.cc:359] Metadata directory not provided
I20260811 09:11:24.290642 24540 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:11:24.296983 24540 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 09:11:24.302819 24571 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:11:24.304605 24540 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 09:11:24.304814 24540 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/master-0/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/master-0/wal
uuid: "adc54f73b9dc4ef09d9ede64155cd1e5"
format_stamp: "Formatted at 2026-08-11 09:11:17 on dist-test-slave-xt4k"
server_key: "546556857f41fa52df510ea4579ae8bf"
server_key_iv: "56d120bb3d2faf25c7812b93fdb857ed"
server_key_version: "encryptionkey@0"
I20260811 09:11:24.305889 24540 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:11:24.322481 24540 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:11:24.327831 24540 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:11:24.328132 24540 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:11:24.348634 24540 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.190:36117
I20260811 09:11:24.348652 24664 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.190:36117 every 8 connection(s)
I20260811 09:11:24.350265 24540 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
I20260811 09:11:24.357815 24665 sys_catalog.cc:263] Verifying existing consensus state
I20260811 09:11:24.360438 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 24540
I20260811 09:11:24.361393 24665 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P adc54f73b9dc4ef09d9ede64155cd1e5: Bootstrap starting.
I20260811 09:11:24.361582 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.154.129:44993
--local_ip_for_outbound_sockets=127.28.154.129
--tserver_master_addrs=127.28.154.190:36117
--webserver_port=40307
--webserver_interface=127.28.154.129
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.129.keytab
--principal=kudu/127.28.154.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:32851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 09:11:24.382329 24665 log.cc:824] T 00000000000000000000000000000000 P adc54f73b9dc4ef09d9ede64155cd1e5: Log is configured to *not* fsync() on all Append() calls
I20260811 09:11:24.401418 24665 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P adc54f73b9dc4ef09d9ede64155cd1e5: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 09:11:24.402010 24665 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P adc54f73b9dc4ef09d9ede64155cd1e5: Bootstrap complete.
I20260811 09:11:24.412271 24665 raft_consensus.cc:348] T 00000000000000000000000000000000 P adc54f73b9dc4ef09d9ede64155cd1e5 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adc54f73b9dc4ef09d9ede64155cd1e5" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 36117 } }
I20260811 09:11:24.413292 24665 raft_consensus.cc:729] T 00000000000000000000000000000000 P adc54f73b9dc4ef09d9ede64155cd1e5 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: adc54f73b9dc4ef09d9ede64155cd1e5, State: Initialized, Role: FOLLOWER
I20260811 09:11:24.413991 24665 consensus_queue.cc:260] T 00000000000000000000000000000000 P adc54f73b9dc4ef09d9ede64155cd1e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adc54f73b9dc4ef09d9ede64155cd1e5" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 36117 } }
I20260811 09:11:24.414227 24665 raft_consensus.cc:388] T 00000000000000000000000000000000 P adc54f73b9dc4ef09d9ede64155cd1e5 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:11:24.414348 24665 raft_consensus.cc:482] T 00000000000000000000000000000000 P adc54f73b9dc4ef09d9ede64155cd1e5 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:11:24.414523 24665 raft_consensus.cc:3037] T 00000000000000000000000000000000 P adc54f73b9dc4ef09d9ede64155cd1e5 [term 2 FOLLOWER]: Advancing to term 3
I20260811 09:11:24.418540 24665 raft_consensus.cc:504] T 00000000000000000000000000000000 P adc54f73b9dc4ef09d9ede64155cd1e5 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adc54f73b9dc4ef09d9ede64155cd1e5" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 36117 } }
I20260811 09:11:24.419085 24665 leader_election.cc:302] T 00000000000000000000000000000000 P adc54f73b9dc4ef09d9ede64155cd1e5 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: adc54f73b9dc4ef09d9ede64155cd1e5; no voters:
I20260811 09:11:24.419765 24665 leader_election.cc:288] T 00000000000000000000000000000000 P adc54f73b9dc4ef09d9ede64155cd1e5 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260811 09:11:24.419994 24671 raft_consensus.cc:2781] T 00000000000000000000000000000000 P adc54f73b9dc4ef09d9ede64155cd1e5 [term 3 FOLLOWER]: Leader election won for term 3
I20260811 09:11:24.420945 24671 raft_consensus.cc:686] T 00000000000000000000000000000000 P adc54f73b9dc4ef09d9ede64155cd1e5 [term 3 LEADER]: Becoming Leader. State: Replica: adc54f73b9dc4ef09d9ede64155cd1e5, State: Running, Role: LEADER
I20260811 09:11:24.421613 24671 consensus_queue.cc:237] T 00000000000000000000000000000000 P adc54f73b9dc4ef09d9ede64155cd1e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adc54f73b9dc4ef09d9ede64155cd1e5" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 36117 } }
I20260811 09:11:24.421957 24665 sys_catalog.cc:564] T 00000000000000000000000000000000 P adc54f73b9dc4ef09d9ede64155cd1e5 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:11:24.424898 24675 sys_catalog.cc:455] T 00000000000000000000000000000000 P adc54f73b9dc4ef09d9ede64155cd1e5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader adc54f73b9dc4ef09d9ede64155cd1e5. Latest consensus state: current_term: 3 leader_uuid: "adc54f73b9dc4ef09d9ede64155cd1e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adc54f73b9dc4ef09d9ede64155cd1e5" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 36117 } } }
I20260811 09:11:24.425292 24675 sys_catalog.cc:458] T 00000000000000000000000000000000 P adc54f73b9dc4ef09d9ede64155cd1e5 [sys.catalog]: This master's current role is: LEADER
I20260811 09:11:24.426126 24673 sys_catalog.cc:455] T 00000000000000000000000000000000 P adc54f73b9dc4ef09d9ede64155cd1e5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "adc54f73b9dc4ef09d9ede64155cd1e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adc54f73b9dc4ef09d9ede64155cd1e5" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 36117 } } }
I20260811 09:11:24.426772 24673 sys_catalog.cc:458] T 00000000000000000000000000000000 P adc54f73b9dc4ef09d9ede64155cd1e5 [sys.catalog]: This master's current role is: LEADER
I20260811 09:11:24.433785 24690 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:11:24.442695 24690 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:11:24.444576 24690 catalog_manager.cc:1259] Loaded cluster ID: f632f10c6dc34ca093d24d59728d1a6b
I20260811 09:11:24.444705 24690 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260811 09:11:24.447126 24690 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
I20260811 09:11:24.449679 24716 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
@ 0x7efe898aa9ad google::LogMessage::Fail() at ??:0
@ 0x7efe898ad88a google::LogMessage::SendToLog() at ??:0
@ 0x7efe898aa4de google::LogMessage::Flush() at ??:0
@ 0x7efe898aba19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7efe9f2dd512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7efe9f2da976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7efe9f362610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7efe9f362439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7efe8c4f6345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7efe8c4f991c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7efe8c4f9799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7efe8c4d4eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7efe8f7ae6db start_thread at ??:0
@ 0x7efe8874461f clone at ??:0
W20260811 09:11:24.811205 24668 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:11:24.811522 24668 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:11:24.811633 24668 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:11:24.819514 24668 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:11:24.819654 24668 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:11:24.819787 24668 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.154.129
I20260811 09:11:24.831207 24668 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:32851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.129.keytab
--principal=kudu/127.28.154.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.154.129:44993
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.154.129
--webserver_port=40307
--tserver_master_addrs=127.28.154.190:36117
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.154.129
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:11:24.832553 24668 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:11:24.833835 24668 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:11:24.847918 24753 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:11:24.847831 24760 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:11:24.847890 24754 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:11:24.848834 24668 server_base.cc:1034] running on GCE node
I20260811 09:11:24.849550 24668 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:11:24.850600 24668 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:11:24.851831 24668 hybrid_clock.cc:648] HybridClock initialized: now 1786439484851771 us; error 47 us; skew 500 ppm
I20260811 09:11:24.852186 24668 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:11:24 dist-test-slave-xt4k krb5kdc[22586](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439484, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:11:24.855468 24668 init.cc:377] Logged in from keytab as kudu/127.28.154.129@KRBTEST.COM (short username kudu)
I20260811 09:11:24.857255 24668 webserver.cc:460] Webserver started at http://127.28.154.129:40307/ using document root <none> and password file <none>
I20260811 09:11:24.858053 24668 fs_manager.cc:359] Metadata directory not provided
I20260811 09:11:24.858198 24668 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:11:24.863998 24668 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 09:11:24.867069 24771 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:11:24.868657 24668 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 09:11:24.868920 24668 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-0/wal
uuid: "e04ee1cf144a4460abe550ff84ff1b29"
format_stamp: "Formatted at 2026-08-11 09:11:18 on dist-test-slave-xt4k"
server_key: "c43fce2f65395065e27c92a5d3d25262"
server_key_iv: "5b5b0ddc94a7f4ad14f8de25489600d0"
server_key_version: "encryptionkey@0"
I20260811 09:11:24.869961 24668 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:11:24.897884 24668 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:11:24.902856 24668 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:11:24.903172 24668 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:11:24.904280 24668 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:11:24.905762 24668 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:11:24.905864 24668 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:11:24.905979 24668 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:11:24.906061 24668 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:11:24.941308 24668 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.129:44993
I20260811 09:11:24.941398 24991 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.129:44993 every 8 connection(s)
I20260811 09:11:24.942821 24668 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-0/data/info.pb
I20260811 09:11:24.948218 24995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:24.946107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:36117 (local address 127.28.154.129:38915)
0811 09:11:24.947036 (+ 929us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:24.947310 (+ 274us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:36117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":656,"thread_start_us":333,"threads_started":1}
W20260811 09:11:24.949110 24992 heartbeater.cc:644] Failed to heartbeat to 127.28.154.190:36117 (0 consecutive failures): Network error: Failed to ping master at 127.28.154.190:36117: Client connection negotiation failed: client connection to 127.28.154.190:36117: connect: Connection refused (error 111)
I20260811 09:11:24.950002 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 24668
I20260811 09:11:24.950142 24995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:24.949706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:36117 (local address 127.28.154.129:52849)
0811 09:11:24.949897 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:24.949992 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:36117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260811 09:11:24.950874 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.154.130:41151
--local_ip_for_outbound_sockets=127.28.154.130
--tserver_master_addrs=127.28.154.190:36117
--webserver_port=34149
--webserver_interface=127.28.154.130
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.130.keytab
--principal=kudu/127.28.154.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:32851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 09:11:24.951156 24995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:24.950755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:36117 (local address 127.28.154.129:35287)
0811 09:11:24.950920 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:24.951006 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:36117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260811 09:11:24.951421 24992 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 09:11:25.321427 24996 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:11:25.321754 24996 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:11:25.321851 24996 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:11:25.329435 24996 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:11:25.329587 24996 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:11:25.329707 24996 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.154.130
I20260811 09:11:25.339708 24996 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:32851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.130.keytab
--principal=kudu/127.28.154.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.154.130:41151
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.154.130
--webserver_port=34149
--tserver_master_addrs=127.28.154.190:36117
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.154.130
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:11:25.341120 24996 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:11:25.342461 24996 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:11:25.356581 25002 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:11:25.356664 25003 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:11:25.356609 25006 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:11:25.358319 24996 server_base.cc:1034] running on GCE node
I20260811 09:11:25.359035 24996 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:11:25.360059 24996 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:11:25.361299 24996 hybrid_clock.cc:648] HybridClock initialized: now 1786439485361244 us; error 36 us; skew 500 ppm
I20260811 09:11:25.361663 24996 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:11:25 dist-test-slave-xt4k krb5kdc[22586](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439485, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:11:25.365131 24996 init.cc:377] Logged in from keytab as kudu/127.28.154.130@KRBTEST.COM (short username kudu)
I20260811 09:11:25.366780 24996 webserver.cc:460] Webserver started at http://127.28.154.130:34149/ using document root <none> and password file <none>
I20260811 09:11:25.367492 24996 fs_manager.cc:359] Metadata directory not provided
I20260811 09:11:25.367606 24996 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:11:25.373510 24996 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 09:11:25.376793 25016 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:11:25.378306 24996 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 09:11:25.378475 24996 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-1/wal
uuid: "048c18609b164ee793f0e33a9ac111a7"
format_stamp: "Formatted at 2026-08-11 09:11:18 on dist-test-slave-xt4k"
server_key: "63e41ac7b26e1cd21bd24dd5da3362e8"
server_key_iv: "a4abc8b6a2eefce27f8af3025ab6be4d"
server_key_version: "encryptionkey@0"
I20260811 09:11:25.379521 24996 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:11:25.401473 24996 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:11:25.405984 24996 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:11:25.406253 24996 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:11:25.407343 24996 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:11:25.408845 24996 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:11:25.408942 24996 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:11:25.409053 24996 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:11:25.409135 24996 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:11:25.446221 24996 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.130:41151
I20260811 09:11:25.446306 25234 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.130:41151 every 8 connection(s)
I20260811 09:11:25.448189 24996 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-1/data/info.pb
I20260811 09:11:25.453310 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 24996
I20260811 09:11:25.454066 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.154.131:35775
--local_ip_for_outbound_sockets=127.28.154.131
--tserver_master_addrs=127.28.154.190:36117
--webserver_port=39891
--webserver_interface=127.28.154.131
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.131.keytab
--principal=kudu/127.28.154.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:32851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 09:11:25.455018 25239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:25.452473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:36117 (local address 127.28.154.130:34029)
0811 09:11:25.453508 (+ 1035us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:25.453816 (+ 308us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:36117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":631,"thread_start_us":381,"threads_started":1}
W20260811 09:11:25.455775 25235 heartbeater.cc:644] Failed to heartbeat to 127.28.154.190:36117 (0 consecutive failures): Network error: Failed to ping master at 127.28.154.190:36117: Client connection negotiation failed: client connection to 127.28.154.190:36117: connect: Connection refused (error 111)
I20260811 09:11:25.456774 25239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:25.456394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:36117 (local address 127.28.154.130:48297)
0811 09:11:25.456556 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:25.456643 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:36117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260811 09:11:25.457711 25239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:25.457337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:36117 (local address 127.28.154.130:46657)
0811 09:11:25.457490 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:25.457583 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:36117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260811 09:11:25.457985 25235 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 09:11:25.799317 25243 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:11:25.799641 25243 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:11:25.799762 25243 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:11:25.807615 25243 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:11:25.807760 25243 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:11:25.807912 25243 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.154.131
I20260811 09:11:25.818357 25243 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:32851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.131.keytab
--principal=kudu/127.28.154.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.154.131:35775
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.154.131
--webserver_port=39891
--tserver_master_addrs=127.28.154.190:36117
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.154.131
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:11:25.819753 25243 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:11:25.821102 25243 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:11:25.835216 25253 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:11:25.835206 25258 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:11:25.835556 25251 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:11:25.836158 25243 server_base.cc:1034] running on GCE node
I20260811 09:11:25.836946 25243 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:11:25.837955 25243 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:11:25.839172 25243 hybrid_clock.cc:648] HybridClock initialized: now 1786439485839107 us; error 52 us; skew 500 ppm
I20260811 09:11:25.839515 25243 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:11:25 dist-test-slave-xt4k krb5kdc[22586](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439485, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:11:25.842814 25243 init.cc:377] Logged in from keytab as kudu/127.28.154.131@KRBTEST.COM (short username kudu)
I20260811 09:11:25.844442 25243 webserver.cc:460] Webserver started at http://127.28.154.131:39891/ using document root <none> and password file <none>
I20260811 09:11:25.845155 25243 fs_manager.cc:359] Metadata directory not provided
I20260811 09:11:25.845278 25243 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:11:25.851235 25243 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 09:11:25.854537 25271 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:11:25.856201 25243 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 09:11:25.856488 25243 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-2/wal
uuid: "699b1810d51a439684efddb537a4348e"
format_stamp: "Formatted at 2026-08-11 09:11:19 on dist-test-slave-xt4k"
server_key: "b375fe65da0c0f7e44fafa54c751e21c"
server_key_iv: "2fbd477725914d6faa1482a07ac32eb2"
server_key_version: "encryptionkey@0"
I20260811 09:11:25.857600 25243 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:11:25.883500 25243 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:11:25.888207 25243 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:11:25.888521 25243 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:11:25.889614 25243 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:11:25.891139 25243 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:11:25.891254 25243 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:11:25.891369 25243 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:11:25.891443 25243 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:11:25.925590 25243 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.131:35775
I20260811 09:11:25.925678 25468 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.131:35775 every 8 connection(s)
I20260811 09:11:25.927064 25243 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786439250192320-29290-0/minicluster-data/ts-2/data/info.pb
I20260811 09:11:25.931981 25474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:25.930122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:36117 (local address 127.28.154.131:58167)
0811 09:11:25.930886 (+ 764us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:25.931124 (+ 238us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:36117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":306,"threads_started":1}
W20260811 09:11:25.932725 25469 heartbeater.cc:644] Failed to heartbeat to 127.28.154.190:36117 (0 consecutive failures): Network error: Failed to ping master at 127.28.154.190:36117: Client connection negotiation failed: client connection to 127.28.154.190:36117: connect: Connection refused (error 111)
I20260811 09:11:25.933949 25474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:25.933479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:36117 (local address 127.28.154.131:43941)
0811 09:11:25.933659 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:25.933828 (+ 169us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:36117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260811 09:11:25.934942 25474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:25.934584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:36117 (local address 127.28.154.131:50227)
0811 09:11:25.934743 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:25.934825 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:36117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260811 09:11:25.935230 25469 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 09:11:25.935216 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 25243
I20260811 09:11:25.952876 25476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:25.951982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:36117 (local address 127.28.154.129:42029)
0811 09:11:25.952499 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:25.952625 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:36117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":228,"threads_started":1}
I20260811 09:11:26.459350 25478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:26.458518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:36117 (local address 127.28.154.130:44275)
0811 09:11:26.458985 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:26.459113 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:36117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":219,"threads_started":1}
I20260811 09:11:26.936723 25481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:26.935833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:36117 (local address 127.28.154.131:57157)
0811 09:11:26.936323 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:26.936483 (+ 160us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:36117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":247,"threads_started":1}
I20260811 09:11:26.954445 25484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:26.953742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:36117 (local address 127.28.154.129:59371)
0811 09:11:26.954193 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:26.954277 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:36117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":209,"threads_started":1}
I20260811 09:11:27.461386 25488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:27.460497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:36117 (local address 127.28.154.130:45097)
0811 09:11:27.461098 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:27.461189 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:36117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":347,"threads_started":1}
I20260811 09:11:27.938515 25489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:27.937702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:36117 (local address 127.28.154.131:56237)
0811 09:11:27.938228 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:27.938317 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:36117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":246,"threads_started":1}
I20260811 09:11:27.956168 25490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:27.955366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:36117 (local address 127.28.154.129:32779)
0811 09:11:27.955881 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:27.955980 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:36117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":294,"threads_started":1}
I20260811 09:11:28.463290 25491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:28.462495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:36117 (local address 127.28.154.130:51431)
0811 09:11:28.462999 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:28.463104 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:36117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":251,"threads_started":1}
I20260811 09:11:28.940237 25496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:28.939459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:36117 (local address 127.28.154.131:33765)
0811 09:11:28.939940 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:28.940028 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:36117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":259,"threads_started":1}
I20260811 09:11:28.958113 25497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:28.957239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:36117 (local address 127.28.154.129:42065)
0811 09:11:28.957825 (+ 586us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:28.957920 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:36117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":669,"threads_started":1}
I20260811 09:11:29.465049 25499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:29.464234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:36117 (local address 127.28.154.130:39503)
0811 09:11:29.464739 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:29.464847 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:36117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":258,"threads_started":1}
I20260811 09:11:29.942178 25500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:29.941251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:36117 (local address 127.28.154.131:42971)
0811 09:11:29.941835 (+ 584us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:29.941971 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:36117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":277,"threads_started":1}
I20260811 09:11:29.960167 25501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:29.959161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:36117 (local address 127.28.154.129:60449)
0811 09:11:29.959872 (+ 711us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:29.959982 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:36117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":299,"threads_started":1}
I20260811 09:11:30.466884 25504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:30.466017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:36117 (local address 127.28.154.130:36435)
0811 09:11:30.466503 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:30.466611 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:36117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":254,"threads_started":1}
I20260811 09:11:30.944022 25505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:30.943181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:36117 (local address 127.28.154.131:53891)
0811 09:11:30.943729 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:30.943835 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:36117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":429,"thread_start_us":287,"threads_started":1}
I20260811 09:11:30.961875 25507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:30.961088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:36117 (local address 127.28.154.129:44673)
0811 09:11:30.961590 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:30.961690 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:36117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":279,"threads_started":1}
I20260811 09:11:31.468842 25510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:31.468006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:36117 (local address 127.28.154.130:34183)
0811 09:11:31.468485 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:31.468625 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:36117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":227,"threads_started":1}
I20260811 09:11:31.945708 25513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:31.944949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:36117 (local address 127.28.154.131:53965)
0811 09:11:31.945426 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:31.945535 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:36117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":237,"threads_started":1}
I20260811 09:11:31.963999 25515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:31.963055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:36117 (local address 127.28.154.129:41369)
0811 09:11:31.963611 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:31.963733 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:36117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":267,"threads_started":1}
I20260811 09:11:32.470708 25517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:32.469855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:36117 (local address 127.28.154.130:42975)
0811 09:11:32.470385 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:32.470465 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:36117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":237,"threads_started":1}
I20260811 09:11:32.947463 25518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:32.946683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:36117 (local address 127.28.154.131:32897)
0811 09:11:32.947174 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:32.947258 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:36117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":205,"threads_started":1}
I20260811 09:11:32.965677 25520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:32.964920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:36117 (local address 127.28.154.129:57079)
0811 09:11:32.965427 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:32.965514 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:36117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":268,"threads_started":1}
I20260811 09:11:33.472769 25521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:33.471852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:36117 (local address 127.28.154.130:44269)
0811 09:11:33.472450 (+ 598us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:33.472553 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:36117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":308,"threads_started":1}
I20260811 09:11:33.949146 25525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:33.948366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:36117 (local address 127.28.154.131:34755)
0811 09:11:33.948860 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:33.948948 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:36117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":222,"threads_started":1}
I20260811 09:11:33.967415 25526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:33.966561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:36117 (local address 127.28.154.129:42833)
0811 09:11:33.967140 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:33.967232 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:36117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":282,"threads_started":1}
I20260811 09:11:34.474680 25527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:34.473804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:36117 (local address 127.28.154.130:45901)
0811 09:11:34.474369 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:34.474471 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:36117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":305,"threads_started":1}
I20260811 09:11:34.950964 25530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:34.950126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:36117 (local address 127.28.154.131:53269)
0811 09:11:34.950646 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:34.950755 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:36117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":243,"threads_started":1}
I20260811 09:11:34.969188 25532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:34.968489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:36117 (local address 127.28.154.129:47921)
0811 09:11:34.968937 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:34.969026 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:36117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":236,"threads_started":1}
I20260811 09:11:35.476255 25533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:35.475552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:36117 (local address 127.28.154.130:52129)
0811 09:11:35.475987 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:35.476091 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:36117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":238,"threads_started":1}
I20260811 09:11:35.952852 25534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:35.952050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:36117 (local address 127.28.154.131:49073)
0811 09:11:35.952579 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:35.952678 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:36117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":415,"thread_start_us":294,"threads_started":1}
I20260811 09:11:35.970932 25539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:35.970148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:36117 (local address 127.28.154.129:59151)
0811 09:11:35.970614 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:35.970724 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:36117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":227,"threads_started":1}
I20260811 09:11:36.477877 25541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:36.477093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:36117 (local address 127.28.154.130:46975)
0811 09:11:36.477558 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:36.477654 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:36117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":234,"threads_started":1}
I20260811 09:11:36.954555 25544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:36.953767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:36117 (local address 127.28.154.131:52479)
0811 09:11:36.954260 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:36.954363 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:36117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":264,"threads_started":1}
I20260811 09:11:36.972817 25545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:36.972007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:36117 (local address 127.28.154.129:41961)
0811 09:11:36.972547 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:36.972639 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:36117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":262,"threads_started":1}
I20260811 09:11:37.479723 25547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:37.478945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:36117 (local address 127.28.154.130:43401)
0811 09:11:37.479452 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:37.479555 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:36117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":257,"threads_started":1}
I20260811 09:11:37.956364 25550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:37.955559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:36117 (local address 127.28.154.131:37031)
0811 09:11:37.956077 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:37.956168 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:36117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":258,"threads_started":1}
I20260811 09:11:37.974786 25551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:37.973951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:36117 (local address 127.28.154.129:37133)
0811 09:11:37.974482 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:37.974584 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:36117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":415,"thread_start_us":260,"threads_started":1}
I20260811 09:11:38.481449 25552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:38.480664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:36117 (local address 127.28.154.130:40919)
0811 09:11:38.481161 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:38.481259 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:36117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":240,"threads_started":1}
I20260811 09:11:38.958329 25555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:38.957421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:36117 (local address 127.28.154.131:51893)
0811 09:11:38.958016 (+ 595us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:38.958121 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:36117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":318,"threads_started":1}
I20260811 09:11:38.976557 25556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:38.975760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:36117 (local address 127.28.154.129:40613)
0811 09:11:38.976235 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:38.976320 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:36117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":256,"threads_started":1}
I20260811 09:11:39.483299 25558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:39.482408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:36117 (local address 127.28.154.130:36661)
0811 09:11:39.482935 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:39.483052 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:36117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":257,"threads_started":1}
I20260811 09:11:39.960153 25560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:39.959261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:36117 (local address 127.28.154.131:51661)
0811 09:11:39.959854 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:39.959957 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:36117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":261,"threads_started":1}
I20260811 09:11:39.978463 25562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:39.977636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:36117 (local address 127.28.154.129:48995)
0811 09:11:39.978144 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:39.978266 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:36117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":283,"threads_started":1}
I20260811 09:11:40.485268 25563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:40.484455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:36117 (local address 127.28.154.130:52981)
0811 09:11:40.484941 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:40.485070 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:36117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":238,"threads_started":1}
I20260811 09:11:40.939086 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 24668
I20260811 09:11:40.962458 25569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:40.961432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:36117 (local address 127.28.154.131:52643)
0811 09:11:40.962146 (+ 714us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:40.962262 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:36117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":575,"thread_start_us":423,"threads_started":1}
I20260811 09:11:40.963352 25564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:40.940556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59872 (local address 127.28.154.129:44993)
0811 09:11:40.941095 (+ 539us) server_negotiation.cc:206] Beginning negotiation
0811 09:11:40.941111 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 09:11:40.941152 (+ 41us) server_negotiation.cc:407] Connection header received
0811 09:11:40.942808 (+ 1656us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:11:40.942822 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:11:40.943409 (+ 587us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:11:40.943795 (+ 386us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:11:40.945346 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:40.946874 (+ 1528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:40.948323 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:40.949112 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:40.949271 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:40.955639 (+ 6368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:11:40.955650 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:11:40.955678 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:11:40.955743 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:11:40.959090 (+ 3347us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:11:40.959929 (+ 839us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:11:40.959938 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:11:40.959946 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:11:40.960172 (+ 226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:11:40.960701 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:11:40.960706 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:11:40.960709 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:11:40.961462 (+ 753us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:11:40.961731 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0811 09:11:40.962491 (+ 760us) server_negotiation.cc:299] Negotiation successful
0811 09:11:40.963067 (+ 576us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":232,"threads_started":1}
I20260811 09:11:40.964869 24976 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:11:41.049304 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 24668
I20260811 09:11:41.076882 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 24996
I20260811 09:11:41.095239 25574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:41.077863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58728 (local address 127.28.154.130:41151)
0811 09:11:41.078474 (+ 611us) server_negotiation.cc:206] Beginning negotiation
0811 09:11:41.078489 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 09:11:41.078526 (+ 37us) server_negotiation.cc:407] Connection header received
0811 09:11:41.080109 (+ 1583us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:11:41.080118 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:11:41.080590 (+ 472us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:11:41.080915 (+ 325us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:11:41.082443 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:41.083770 (+ 1327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:41.085163 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:41.085869 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:41.086048 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:41.088755 (+ 2707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:11:41.088764 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:11:41.088780 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:11:41.088819 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:11:41.091667 (+ 2848us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:11:41.092530 (+ 863us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:11:41.092540 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:11:41.092550 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:11:41.092737 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:11:41.093127 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:11:41.093131 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:11:41.093133 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:11:41.093686 (+ 553us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:11:41.094011 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0811 09:11:41.094473 (+ 462us) server_negotiation.cc:299] Negotiation successful
0811 09:11:41.094935 (+ 462us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":417,"thread_start_us":280,"threads_started":1}
I20260811 09:11:41.096791 25228 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:11:41.180758 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 24996
I20260811 09:11:41.209692 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 25243
I20260811 09:11:41.228387 25577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:41.210702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38714 (local address 127.28.154.131:35775)
0811 09:11:41.211296 (+ 594us) server_negotiation.cc:206] Beginning negotiation
0811 09:11:41.211311 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 09:11:41.211357 (+ 46us) server_negotiation.cc:407] Connection header received
0811 09:11:41.213021 (+ 1664us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:11:41.213032 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:11:41.213465 (+ 433us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:11:41.213763 (+ 298us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:11:41.215150 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:41.216780 (+ 1630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:41.218068 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:41.218797 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:41.218939 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:41.221886 (+ 2947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:11:41.221895 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:11:41.221913 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:11:41.221967 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:11:41.225083 (+ 3116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:11:41.225893 (+ 810us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:11:41.225901 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:11:41.225909 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:11:41.226104 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:11:41.226450 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:11:41.226452 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:11:41.226454 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:11:41.227007 (+ 553us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:11:41.227245 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0811 09:11:41.227635 (+ 390us) server_negotiation.cc:299] Negotiation successful
0811 09:11:41.228050 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":427,"thread_start_us":281,"threads_started":1}
I20260811 09:11:41.229796 25462 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:11:41.313745 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 25243
2026-08-11T09:11:41Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (25708 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:11:41 dist-test-slave-xt4k krb5kdc[25585](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:11:41 dist-test-slave-xt4k krb5kdc[25585](info): set up 2 sockets
Aug 11 09:11:41 dist-test-slave-xt4k krb5kdc[25585](info): commencing operation
krb5kdc: starting...
W20260811 09:11:42.618711 29290 mini_kdc.cc:121] Time spent starting KDC: real 1.257s user 0.001s sys 0.051s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/krb5kdc/test-user.keytab.
Aug 11 09:11:42 dist-test-slave-xt4k krb5kdc[25585](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439502, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:11:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:11:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.154.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.154.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.154.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab.
Entry for principal kudu/127.28.154.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab.
I20260811 09:11:42.927834 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.154.190:40403
--webserver_interface=127.28.154.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab
--principal=kudu/127.28.154.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:38289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.154.190:40403
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:11:43.272624 25604 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:11:43.272958 25604 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:11:43.273095 25604 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:11:43.280959 25604 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 09:11:43.281111 25604 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 09:11:43.281180 25604 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 09:11:43.281255 25604 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:11:43.281332 25604 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:11:43.281430 25604 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:11:43.281489 25604 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:11:43.281574 25604 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:11:43.292281 25604 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:38289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.28.154.190:40403
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab
--principal=kudu/127.28.154.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.154.190:40403
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.154.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:11:43.293653 25604 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:11:43.295058 25604 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:11:43.307205 25616 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:11:43.307152 25610 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:11:43.307126 25612 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:11:43.307968 25604 server_base.cc:1034] running on GCE node
I20260811 09:11:43.308851 25604 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:11:43.310341 25604 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:11:43.311573 25604 hybrid_clock.cc:648] HybridClock initialized: now 1786439503311502 us; error 44 us; skew 500 ppm
I20260811 09:11:43.311902 25604 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:11:43 dist-test-slave-xt4k krb5kdc[25585](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439503, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:11:43.315162 25604 init.cc:377] Logged in from keytab as kudu/127.28.154.190@KRBTEST.COM (short username kudu)
I20260811 09:11:43.316764 25604 webserver.cc:460] Webserver started at http://127.28.154.190:44341/ using document root <none> and password file <none>
I20260811 09:11:43.317480 25604 fs_manager.cc:359] Metadata directory not provided
I20260811 09:11:43.317612 25604 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:11:43.317942 25604 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:11:43.321100 25604 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/master-0/data/instance:
uuid: "b740e22f6c334fb38cdc579e01606268"
format_stamp: "Formatted at 2026-08-11 09:11:43 on dist-test-slave-xt4k"
server_key: "ada7b95799df2469c5c6b10b1a71d5d8"
server_key_iv: "76280d86cd4f431e0dd06f2758862612"
server_key_version: "encryptionkey@0"
I20260811 09:11:43.321924 25604 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/master-0/wal/instance:
uuid: "b740e22f6c334fb38cdc579e01606268"
format_stamp: "Formatted at 2026-08-11 09:11:43 on dist-test-slave-xt4k"
server_key: "ada7b95799df2469c5c6b10b1a71d5d8"
server_key_iv: "76280d86cd4f431e0dd06f2758862612"
server_key_version: "encryptionkey@0"
I20260811 09:11:43.327450 25604 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260811 09:11:43.331272 25628 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:11:43.332747 25604 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 09:11:43.332912 25604 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/master-0/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/master-0/wal
uuid: "b740e22f6c334fb38cdc579e01606268"
format_stamp: "Formatted at 2026-08-11 09:11:43 on dist-test-slave-xt4k"
server_key: "ada7b95799df2469c5c6b10b1a71d5d8"
server_key_iv: "76280d86cd4f431e0dd06f2758862612"
server_key_version: "encryptionkey@0"
I20260811 09:11:43.333125 25604 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:11:43.356899 25604 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:11:43.361281 25604 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:11:43.361542 25604 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:11:43.379630 25604 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.190:40403
I20260811 09:11:43.379654 25726 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.190:40403 every 8 connection(s)
I20260811 09:11:43.381004 25604 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
I20260811 09:11:43.384867 25727 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:11:43.387058 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 25604
I20260811 09:11:43.387253 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/master-0/wal/instance
I20260811 09:11:43.387957 29290 external_mini_cluster.cc:1428] Setting key 878d937db3f50e43efec9b21305bfff2
I20260811 09:11:43.397220 25727 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b740e22f6c334fb38cdc579e01606268: Bootstrap starting.
I20260811 09:11:43.401252 25727 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b740e22f6c334fb38cdc579e01606268: Neither blocks nor log segments found. Creating new log.
I20260811 09:11:43.402653 25727 log.cc:824] T 00000000000000000000000000000000 P b740e22f6c334fb38cdc579e01606268: Log is configured to *not* fsync() on all Append() calls
Aug 11 09:11:43 dist-test-slave-xt4k krb5kdc[25585](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439502, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.154.190@KRBTEST.COM
I20260811 09:11:43.406165 25727 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b740e22f6c334fb38cdc579e01606268: No bootstrap required, opened a new log
I20260811 09:11:43.413618 25730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:43.390135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33886 (local address 127.28.154.190:40403)
0811 09:11:43.390923 (+ 788us) server_negotiation.cc:206] Beginning negotiation
0811 09:11:43.390935 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 09:11:43.390977 (+ 42us) server_negotiation.cc:407] Connection header received
0811 09:11:43.392776 (+ 1799us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:11:43.392801 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:11:43.393194 (+ 393us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:11:43.393521 (+ 327us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:11:43.395464 (+ 1943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:43.397111 (+ 1647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:43.398997 (+ 1886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:43.399734 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:43.399866 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:43.405211 (+ 5345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:11:43.405219 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:11:43.405236 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:11:43.405275 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:11:43.408544 (+ 3269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:11:43.409779 (+ 1235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:11:43.409796 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:11:43.409807 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:11:43.410025 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:11:43.410479 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:11:43.410483 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:11:43.410486 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:11:43.411094 (+ 608us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:11:43.411338 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0811 09:11:43.411962 (+ 624us) server_negotiation.cc:299] Negotiation successful
0811 09:11:43.412483 (+ 521us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":582,"thread_start_us":349,"threads_started":1}
I20260811 09:11:43.415910 25727 raft_consensus.cc:348] T 00000000000000000000000000000000 P b740e22f6c334fb38cdc579e01606268 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b740e22f6c334fb38cdc579e01606268" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 40403 } }
I20260811 09:11:43.416483 25727 raft_consensus.cc:374] T 00000000000000000000000000000000 P b740e22f6c334fb38cdc579e01606268 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:11:43.416594 25727 raft_consensus.cc:729] T 00000000000000000000000000000000 P b740e22f6c334fb38cdc579e01606268 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b740e22f6c334fb38cdc579e01606268, State: Initialized, Role: FOLLOWER
I20260811 09:11:43.417280 25727 consensus_queue.cc:260] T 00000000000000000000000000000000 P b740e22f6c334fb38cdc579e01606268 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b740e22f6c334fb38cdc579e01606268" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 40403 } }
I20260811 09:11:43.417512 25727 raft_consensus.cc:388] T 00000000000000000000000000000000 P b740e22f6c334fb38cdc579e01606268 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:11:43.417634 25727 raft_consensus.cc:482] T 00000000000000000000000000000000 P b740e22f6c334fb38cdc579e01606268 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:11:43.417789 25727 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b740e22f6c334fb38cdc579e01606268 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:11:43.420429 25727 raft_consensus.cc:504] T 00000000000000000000000000000000 P b740e22f6c334fb38cdc579e01606268 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b740e22f6c334fb38cdc579e01606268" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 40403 } }
I20260811 09:11:43.421060 25727 leader_election.cc:302] T 00000000000000000000000000000000 P b740e22f6c334fb38cdc579e01606268 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b740e22f6c334fb38cdc579e01606268; no voters:
I20260811 09:11:43.421674 25727 leader_election.cc:288] T 00000000000000000000000000000000 P b740e22f6c334fb38cdc579e01606268 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:11:43.421906 25733 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b740e22f6c334fb38cdc579e01606268 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:11:43.422631 25733 raft_consensus.cc:686] T 00000000000000000000000000000000 P b740e22f6c334fb38cdc579e01606268 [term 1 LEADER]: Becoming Leader. State: Replica: b740e22f6c334fb38cdc579e01606268, State: Running, Role: LEADER
I20260811 09:11:43.423194 25733 consensus_queue.cc:237] T 00000000000000000000000000000000 P b740e22f6c334fb38cdc579e01606268 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b740e22f6c334fb38cdc579e01606268" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 40403 } }
I20260811 09:11:43.423810 25727 sys_catalog.cc:564] T 00000000000000000000000000000000 P b740e22f6c334fb38cdc579e01606268 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:11:43.426047 25739 sys_catalog.cc:455] T 00000000000000000000000000000000 P b740e22f6c334fb38cdc579e01606268 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b740e22f6c334fb38cdc579e01606268" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b740e22f6c334fb38cdc579e01606268" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 40403 } } }
I20260811 09:11:43.426431 25739 sys_catalog.cc:458] T 00000000000000000000000000000000 P b740e22f6c334fb38cdc579e01606268 [sys.catalog]: This master's current role is: LEADER
I20260811 09:11:43.426887 25742 sys_catalog.cc:455] T 00000000000000000000000000000000 P b740e22f6c334fb38cdc579e01606268 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b740e22f6c334fb38cdc579e01606268. Latest consensus state: current_term: 1 leader_uuid: "b740e22f6c334fb38cdc579e01606268" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b740e22f6c334fb38cdc579e01606268" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 40403 } } }
I20260811 09:11:43.427222 25742 sys_catalog.cc:458] T 00000000000000000000000000000000 P b740e22f6c334fb38cdc579e01606268 [sys.catalog]: This master's current role is: LEADER
I20260811 09:11:43.427366 25752 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:11:43.434233 25752 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:11:43.443200 25752 catalog_manager.cc:1347] Generated new cluster ID: 959c088f5a144412a35c0e95cbab5cb2
I20260811 09:11:43.443322 25752 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:11:43.471864 25752 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:11:43.473421 25752 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:11:43.481873 25752 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b740e22f6c334fb38cdc579e01606268: Generated new TSK 0
I20260811 09:11:43.482802 25752 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.28.154.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.154.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.154.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.129.keytab.
Entry for principal kudu/127.28.154.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.129.keytab.
I20260811 09:11:43.557045 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.154.129:0
--local_ip_for_outbound_sockets=127.28.154.129
--webserver_interface=127.28.154.129
--webserver_port=0
--tserver_master_addrs=127.28.154.190:40403
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.129.keytab
--principal=kudu/127.28.154.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:38289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:11:43.903456 25778 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:11:43.903774 25778 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:11:43.903903 25778 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:11:43.911453 25778 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:11:43.911597 25778 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:11:43.911746 25778 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.154.129
I20260811 09:11:43.922125 25778 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:38289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.129.keytab
--principal=kudu/127.28.154.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.154.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.154.129
--webserver_port=0
--tserver_master_addrs=127.28.154.190:40403
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.154.129
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:11:43.923604 25778 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:11:43.924998 25778 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:11:43.938831 25792 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:11:43.938794 25798 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:11:43.939292 25793 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:11:43.939669 25778 server_base.cc:1034] running on GCE node
I20260811 09:11:43.940303 25778 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:11:43.941464 25778 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:11:43.942812 25778 hybrid_clock.cc:648] HybridClock initialized: now 1786439503942742 us; error 60 us; skew 500 ppm
I20260811 09:11:43.943156 25778 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:11:43 dist-test-slave-xt4k krb5kdc[25585](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439503, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:11:43.946329 25778 init.cc:377] Logged in from keytab as kudu/127.28.154.129@KRBTEST.COM (short username kudu)
I20260811 09:11:43.947768 25778 webserver.cc:460] Webserver started at http://127.28.154.129:36297/ using document root <none> and password file <none>
I20260811 09:11:43.948480 25778 fs_manager.cc:359] Metadata directory not provided
I20260811 09:11:43.948598 25778 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:11:43.948860 25778 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:11:43.952049 25778 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-0/data/instance:
uuid: "153e73b493bf47509fd161e335705439"
format_stamp: "Formatted at 2026-08-11 09:11:43 on dist-test-slave-xt4k"
server_key: "a08bdbd47d5f29477d745c6e9f160c9d"
server_key_iv: "ae5122a0f882baf48ec9c9e6bd0c8b01"
server_key_version: "encryptionkey@0"
I20260811 09:11:43.952883 25778 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-0/wal/instance:
uuid: "153e73b493bf47509fd161e335705439"
format_stamp: "Formatted at 2026-08-11 09:11:43 on dist-test-slave-xt4k"
server_key: "a08bdbd47d5f29477d745c6e9f160c9d"
server_key_iv: "ae5122a0f882baf48ec9c9e6bd0c8b01"
server_key_version: "encryptionkey@0"
I20260811 09:11:43.958511 25778 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260811 09:11:43.962287 25808 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:11:43.963766 25778 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 09:11:43.963920 25778 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-0/wal
uuid: "153e73b493bf47509fd161e335705439"
format_stamp: "Formatted at 2026-08-11 09:11:43 on dist-test-slave-xt4k"
server_key: "a08bdbd47d5f29477d745c6e9f160c9d"
server_key_iv: "ae5122a0f882baf48ec9c9e6bd0c8b01"
server_key_version: "encryptionkey@0"
I20260811 09:11:43.964118 25778 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:11:43.986773 25778 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:11:43.991364 25778 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:11:43.991671 25778 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:11:43.992925 25778 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:11:43.994616 25778 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:11:43.994697 25778 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:11:43.994796 25778 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:11:43.994839 25778 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:11:44.028046 25778 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.129:41979
I20260811 09:11:44.028129 26018 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.129:41979 every 8 connection(s)
I20260811 09:11:44.029378 25778 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-0/data/info.pb
I20260811 09:11:44.038890 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 25778
I20260811 09:11:44.039098 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-0/wal/instance
I20260811 09:11:44.039772 29290 external_mini_cluster.cc:1428] Setting key 8aa1f1fe5775036d575e7644b53c26b7
Aug 11 09:11:44 dist-test-slave-xt4k krb5kdc[25585](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439503, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.129@KRBTEST.COM for kudu/127.28.154.190@KRBTEST.COM
I20260811 09:11:44.050550 26024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:44.032262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.129:57367 (local address 127.28.154.190:40403)
0811 09:11:44.032744 (+ 482us) server_negotiation.cc:206] Beginning negotiation
0811 09:11:44.032749 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:11:44.033707 (+ 958us) server_negotiation.cc:407] Connection header received
0811 09:11:44.035622 (+ 1915us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:11:44.035628 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:11:44.035711 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:11:44.035822 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:11:44.037486 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:44.038280 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:44.040096 (+ 1816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:44.040659 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:44.040783 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:44.044623 (+ 3840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:11:44.044630 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:11:44.044633 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:11:44.044671 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:11:44.047315 (+ 2644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:11:44.048137 (+ 822us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:11:44.048141 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:11:44.048144 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:11:44.048275 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:11:44.048747 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:11:44.048751 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:11:44.048753 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:11:44.049031 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:11:44.049170 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0811 09:11:44.049932 (+ 762us) server_negotiation.cc:299] Negotiation successful
0811 09:11:44.050182 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365,"thread_start_us":255,"threads_started":1}
I20260811 09:11:44.051522 26025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:44.032835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:40403 (local address 127.28.154.129:57367)
0811 09:11:44.033544 (+ 709us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:44.033582 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 09:11:44.035343 (+ 1761us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:11:44.036081 (+ 738us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:11:44.036089 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:11:44.036680 (+ 591us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:11:44.037303 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:11:44.037323 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:11:44.038536 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:11:44.038541 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:11:44.039939 (+ 1398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:11:44.039953 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:11:44.040900 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:11:44.040908 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:11:44.041153 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:44.041815 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:11:44.041844 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:11:44.044325 (+ 2481us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:11:44.047484 (+ 3159us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:11:44.047495 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:11:44.047516 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:11:44.047999 (+ 483us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:11:44.048404 (+ 405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:11:44.048409 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:11:44.048413 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:11:44.048619 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:11:44.049161 (+ 542us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:11:44.049171 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:11:44.049546 (+ 375us) client_negotiation.cc:769] Sending connection context
0811 09:11:44.049798 (+ 252us) client_negotiation.cc:240] Negotiation successful
0811 09:11:44.050378 (+ 580us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":253,"threads_started":1}
I20260811 09:11:44.053378 26022 heartbeater.cc:347] Connected to a master server at 127.28.154.190:40403
I20260811 09:11:44.053784 26022 heartbeater.cc:464] Registering TS with master...
I20260811 09:11:44.054596 26022 heartbeater.cc:511] Master 127.28.154.190:40403 requested a full tablet report, sending...
I20260811 09:11:44.056906 25657 ts_manager.cc:194] Registered new tserver with Master: 153e73b493bf47509fd161e335705439 (127.28.154.129:41979)
I20260811 09:11:44.058426 25657 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.154.129@KRBTEST.COM'} at 127.28.154.129:57367
WARNING: no policy specified for kudu/127.28.154.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.154.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.154.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.130.keytab.
Entry for principal kudu/127.28.154.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.130.keytab.
I20260811 09:11:44.104547 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.154.130:0
--local_ip_for_outbound_sockets=127.28.154.130
--webserver_interface=127.28.154.130
--webserver_port=0
--tserver_master_addrs=127.28.154.190:40403
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.130.keytab
--principal=kudu/127.28.154.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:38289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:11:44.450456 26031 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:11:44.450765 26031 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:11:44.450861 26031 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:11:44.458510 26031 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:11:44.458657 26031 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:11:44.458775 26031 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.154.130
I20260811 09:11:44.470156 26031 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:38289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.130.keytab
--principal=kudu/127.28.154.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.154.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.154.130
--webserver_port=0
--tserver_master_addrs=127.28.154.190:40403
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.154.130
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:11:44.471514 26031 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:11:44.473405 26031 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:11:44.487170 26039 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:11:44.487447 26047 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:11:44.487149 26038 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:11:44.488287 26031 server_base.cc:1034] running on GCE node
I20260811 09:11:44.488943 26031 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:11:44.489915 26031 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:11:44.491129 26031 hybrid_clock.cc:648] HybridClock initialized: now 1786439504491064 us; error 51 us; skew 500 ppm
I20260811 09:11:44.491457 26031 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:11:44 dist-test-slave-xt4k krb5kdc[25585](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439504, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:11:44.494812 26031 init.cc:377] Logged in from keytab as kudu/127.28.154.130@KRBTEST.COM (short username kudu)
I20260811 09:11:44.496364 26031 webserver.cc:460] Webserver started at http://127.28.154.130:45323/ using document root <none> and password file <none>
I20260811 09:11:44.497071 26031 fs_manager.cc:359] Metadata directory not provided
I20260811 09:11:44.497190 26031 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:11:44.497494 26031 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:11:44.500479 26031 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-1/data/instance:
uuid: "84334312b3254576991f3f49ecc134b1"
format_stamp: "Formatted at 2026-08-11 09:11:44 on dist-test-slave-xt4k"
server_key: "57769100a6c65364fdc9ec81b45f0a3a"
server_key_iv: "eff99c23b261da5a2ed4654f64797ee0"
server_key_version: "encryptionkey@0"
I20260811 09:11:44.501236 26031 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-1/wal/instance:
uuid: "84334312b3254576991f3f49ecc134b1"
format_stamp: "Formatted at 2026-08-11 09:11:44 on dist-test-slave-xt4k"
server_key: "57769100a6c65364fdc9ec81b45f0a3a"
server_key_iv: "eff99c23b261da5a2ed4654f64797ee0"
server_key_version: "encryptionkey@0"
I20260811 09:11:44.506382 26031 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 09:11:44.510043 26061 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:11:44.511363 26031 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 09:11:44.511518 26031 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-1/wal
uuid: "84334312b3254576991f3f49ecc134b1"
format_stamp: "Formatted at 2026-08-11 09:11:44 on dist-test-slave-xt4k"
server_key: "57769100a6c65364fdc9ec81b45f0a3a"
server_key_iv: "eff99c23b261da5a2ed4654f64797ee0"
server_key_version: "encryptionkey@0"
I20260811 09:11:44.511714 26031 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:11:44.538358 26031 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:11:44.542834 26031 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:11:44.543118 26031 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:11:44.544114 26031 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:11:44.545495 26031 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:11:44.545575 26031 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:11:44.545691 26031 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:11:44.545742 26031 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:11:44.577268 26031 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.130:37427
I20260811 09:11:44.577311 26282 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.130:37427 every 8 connection(s)
I20260811 09:11:44.578651 26031 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-1/data/info.pb
I20260811 09:11:44.585163 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 26031
I20260811 09:11:44.585379 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-1/wal/instance
I20260811 09:11:44.585963 29290 external_mini_cluster.cc:1428] Setting key 7d5cbb2a8cec794ed7e3c6ab9e752010
Aug 11 09:11:44 dist-test-slave-xt4k krb5kdc[25585](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439504, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.130@KRBTEST.COM for kudu/127.28.154.190@KRBTEST.COM
I20260811 09:11:44.599640 26288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:44.581468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.130:54127 (local address 127.28.154.190:40403)
0811 09:11:44.581968 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0811 09:11:44.581974 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:11:44.583106 (+ 1132us) server_negotiation.cc:407] Connection header received
0811 09:11:44.585036 (+ 1930us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:11:44.585041 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:11:44.585117 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:11:44.585218 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:11:44.587381 (+ 2163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:44.588224 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:44.589814 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:44.590258 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:44.590370 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:44.593720 (+ 3350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:11:44.593725 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:11:44.593728 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:11:44.593763 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:11:44.596455 (+ 2692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:11:44.597220 (+ 765us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:11:44.597224 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:11:44.597226 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:11:44.597362 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:11:44.597806 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:11:44.597810 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:11:44.597812 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:11:44.598125 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:11:44.598266 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0811 09:11:44.599006 (+ 740us) server_negotiation.cc:299] Negotiation successful
0811 09:11:44.599257 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":358,"thread_start_us":234,"threads_started":1}
I20260811 09:11:44.600669 26289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:44.582067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:40403 (local address 127.28.154.130:54127)
0811 09:11:44.582932 (+ 865us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:44.582972 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 09:11:44.584714 (+ 1742us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:11:44.585509 (+ 795us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:11:44.585520 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:11:44.586403 (+ 883us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:11:44.587199 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:11:44.587227 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:11:44.588402 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:11:44.588406 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:11:44.589670 (+ 1264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:11:44.589682 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:11:44.590493 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:11:44.590502 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:11:44.590715 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:44.591355 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:11:44.591378 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:11:44.593493 (+ 2115us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:11:44.596613 (+ 3120us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:11:44.596624 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:11:44.596639 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:11:44.597088 (+ 449us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:11:44.597470 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:11:44.597475 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:11:44.597478 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:11:44.597689 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:11:44.598262 (+ 573us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:11:44.598270 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:11:44.598630 (+ 360us) client_negotiation.cc:769] Sending connection context
0811 09:11:44.598881 (+ 251us) client_negotiation.cc:240] Negotiation successful
0811 09:11:44.599439 (+ 558us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":531,"thread_start_us":307,"threads_started":1}
I20260811 09:11:44.602145 26286 heartbeater.cc:347] Connected to a master server at 127.28.154.190:40403
I20260811 09:11:44.602490 26286 heartbeater.cc:464] Registering TS with master...
I20260811 09:11:44.603361 26286 heartbeater.cc:511] Master 127.28.154.190:40403 requested a full tablet report, sending...
I20260811 09:11:44.605036 25657 ts_manager.cc:194] Registered new tserver with Master: 84334312b3254576991f3f49ecc134b1 (127.28.154.130:37427)
I20260811 09:11:44.606104 25657 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.154.130@KRBTEST.COM'} at 127.28.154.130:54127
WARNING: no policy specified for kudu/127.28.154.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.154.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.154.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.131.keytab.
Entry for principal kudu/127.28.154.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.131.keytab.
I20260811 09:11:44.652451 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.154.131:0
--local_ip_for_outbound_sockets=127.28.154.131
--webserver_interface=127.28.154.131
--webserver_port=0
--tserver_master_addrs=127.28.154.190:40403
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.131.keytab
--principal=kudu/127.28.154.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:38289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:11:44.999421 26294 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:11:44.999740 26294 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:11:44.999871 26294 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:11:45.007302 26294 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:11:45.007463 26294 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:11:45.007628 26294 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.154.131
I20260811 09:11:45.017956 26294 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:38289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.131.keytab
--principal=kudu/127.28.154.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.154.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.154.131
--webserver_port=0
--tserver_master_addrs=127.28.154.190:40403
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.154.131
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:11:45.019330 26294 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:11:45.020794 26294 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:11:45.034988 26310 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:11:45.035022 26312 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:11:45.035501 26309 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:11:45.036731 26294 server_base.cc:1034] running on GCE node
I20260811 09:11:45.037467 26294 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:11:45.038547 26294 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:11:45.039803 26294 hybrid_clock.cc:648] HybridClock initialized: now 1786439505039733 us; error 52 us; skew 500 ppm
I20260811 09:11:45.040169 26294 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:11:45 dist-test-slave-xt4k krb5kdc[25585](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439505, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:11:45.043593 26294 init.cc:377] Logged in from keytab as kudu/127.28.154.131@KRBTEST.COM (short username kudu)
I20260811 09:11:45.045195 26294 webserver.cc:460] Webserver started at http://127.28.154.131:43089/ using document root <none> and password file <none>
I20260811 09:11:45.045939 26294 fs_manager.cc:359] Metadata directory not provided
I20260811 09:11:45.046062 26294 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:11:45.046373 26294 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:11:45.049460 26294 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-2/data/instance:
uuid: "82c897eb04564e7d8870a909b3082d02"
format_stamp: "Formatted at 2026-08-11 09:11:45 on dist-test-slave-xt4k"
server_key: "a90f2eeafb5bab0abe030a448def00db"
server_key_iv: "6895a616d1b1d17f28b8186124151e9f"
server_key_version: "encryptionkey@0"
I20260811 09:11:45.050254 26294 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-2/wal/instance:
uuid: "82c897eb04564e7d8870a909b3082d02"
format_stamp: "Formatted at 2026-08-11 09:11:45 on dist-test-slave-xt4k"
server_key: "a90f2eeafb5bab0abe030a448def00db"
server_key_iv: "6895a616d1b1d17f28b8186124151e9f"
server_key_version: "encryptionkey@0"
I20260811 09:11:45.055745 26294 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 09:11:45.059481 26328 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:11:45.060947 26294 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 09:11:45.061116 26294 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-2/wal
uuid: "82c897eb04564e7d8870a909b3082d02"
format_stamp: "Formatted at 2026-08-11 09:11:45 on dist-test-slave-xt4k"
server_key: "a90f2eeafb5bab0abe030a448def00db"
server_key_iv: "6895a616d1b1d17f28b8186124151e9f"
server_key_version: "encryptionkey@0"
I20260811 09:11:45.061316 26294 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:11:45.061858 26022 heartbeater.cc:503] Master 127.28.154.190:40403 was elected leader, sending a full tablet report...
I20260811 09:11:45.087911 26294 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:11:45.092687 26294 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:11:45.092975 26294 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:11:45.093997 26294 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:11:45.095403 26294 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:11:45.095536 26294 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260811 09:11:45.095670 26294 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:11:45.095726 26294 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:11:45.128222 26294 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.131:38565
I20260811 09:11:45.128268 26556 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.131:38565 every 8 connection(s)
I20260811 09:11:45.129596 26294 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-2/data/info.pb
I20260811 09:11:45.131913 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 26294
I20260811 09:11:45.132098 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-2/wal/instance
I20260811 09:11:45.132665 29290 external_mini_cluster.cc:1428] Setting key 832504c0d17181209429206ea7c52af1
Aug 11 09:11:45 dist-test-slave-xt4k krb5kdc[25585](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439505, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.131@KRBTEST.COM for kudu/127.28.154.190@KRBTEST.COM
I20260811 09:11:45.151713 26560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:45.132708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.131:40267 (local address 127.28.154.190:40403)
0811 09:11:45.133236 (+ 528us) server_negotiation.cc:206] Beginning negotiation
0811 09:11:45.133243 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:11:45.134437 (+ 1194us) server_negotiation.cc:407] Connection header received
0811 09:11:45.136867 (+ 2430us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:11:45.136871 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:11:45.136941 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:11:45.137050 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:11:45.138881 (+ 1831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:45.139727 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:45.141494 (+ 1767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:45.141946 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:45.142054 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:45.145608 (+ 3554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:11:45.145614 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:11:45.145617 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:11:45.145653 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:11:45.148606 (+ 2953us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:11:45.149379 (+ 773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:11:45.149384 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:11:45.149386 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:11:45.149515 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:11:45.149948 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:11:45.149952 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:11:45.149954 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:11:45.150235 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:11:45.150376 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0811 09:11:45.151086 (+ 710us) server_negotiation.cc:299] Negotiation successful
0811 09:11:45.151314 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":235,"threads_started":1}
I20260811 09:11:45.152714 26561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:45.133350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:40403 (local address 127.28.154.131:40267)
0811 09:11:45.134261 (+ 911us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:45.134324 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0811 09:11:45.136584 (+ 2260us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:11:45.137353 (+ 769us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:11:45.137361 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:11:45.137959 (+ 598us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:11:45.138613 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:11:45.138633 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:11:45.139893 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:11:45.139897 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:11:45.141315 (+ 1418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:11:45.141333 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:11:45.142161 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:11:45.142166 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:11:45.142393 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:45.143055 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:11:45.143078 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:11:45.145363 (+ 2285us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:11:45.148748 (+ 3385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:11:45.148759 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:11:45.148776 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:11:45.149256 (+ 480us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:11:45.149622 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:11:45.149628 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:11:45.149632 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:11:45.149830 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:11:45.150361 (+ 531us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:11:45.150370 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:11:45.150723 (+ 353us) client_negotiation.cc:769] Sending connection context
0811 09:11:45.150968 (+ 245us) client_negotiation.cc:240] Negotiation successful
0811 09:11:45.151534 (+ 566us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":601,"thread_start_us":385,"threads_started":1}
I20260811 09:11:45.154091 26558 heartbeater.cc:347] Connected to a master server at 127.28.154.190:40403
I20260811 09:11:45.154371 26558 heartbeater.cc:464] Registering TS with master...
I20260811 09:11:45.155086 26558 heartbeater.cc:511] Master 127.28.154.190:40403 requested a full tablet report, sending...
I20260811 09:11:45.156554 25657 ts_manager.cc:194] Registered new tserver with Master: 82c897eb04564e7d8870a909b3082d02 (127.28.154.131:38565)
I20260811 09:11:45.157354 25657 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.154.131@KRBTEST.COM'} at 127.28.154.131:40267
I20260811 09:11:45.161564 29290 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 09:11:45.180197 26560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:45.166380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41144 (local address 127.28.154.190:40403)
0811 09:11:45.166561 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0811 09:11:45.166566 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:11:45.166981 (+ 415us) server_negotiation.cc:407] Connection header received
0811 09:11:45.167163 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:11:45.167168 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:11:45.167264 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:11:45.167344 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:11:45.169056 (+ 1712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:45.169848 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:45.171454 (+ 1606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:45.171903 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:45.172041 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:45.174652 (+ 2611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:11:45.174658 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:11:45.174661 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:11:45.174704 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:11:45.177409 (+ 2705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:11:45.178147 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:11:45.178152 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:11:45.178154 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:11:45.178305 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:11:45.178747 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:11:45.178751 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:11:45.178754 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:11:45.179147 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:11:45.179358 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0811 09:11:45.179749 (+ 391us) server_negotiation.cc:299] Negotiation successful
0811 09:11:45.179944 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 09:11:45.184744 25657 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41144:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 09:11:45.191685 25657 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 09:11:45.225219 26580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:45.204230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.130:37427 (local address 127.0.0.1:36344)
0811 09:11:45.205017 (+ 787us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:45.205070 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 09:11:45.205256 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:11:45.218163 (+ 12907us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:11:45.218169 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:11:45.218200 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:11:45.218646 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:11:45.218661 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:11:45.220582 (+ 1921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:11:45.220587 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:11:45.222539 (+ 1952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:11:45.222554 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:11:45.224052 (+ 1498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:11:45.224059 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:11:45.224277 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:45.224397 (+ 120us) client_negotiation.cc:769] Sending connection context
0811 09:11:45.224521 (+ 124us) client_negotiation.cc:240] Negotiation successful
0811 09:11:45.224746 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":500,"mutex_wait_us":6,"thread_start_us":603,"threads_started":1}
I20260811 09:11:45.225683 26581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:45.204428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.131:38565 (local address 127.0.0.1:41928)
0811 09:11:45.216183 (+ 11755us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:45.216202 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 09:11:45.216403 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:11:45.217080 (+ 677us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:11:45.217088 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:11:45.217134 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:11:45.217503 (+ 369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:11:45.217515 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:11:45.219278 (+ 1763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:11:45.219284 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:11:45.221957 (+ 2673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:11:45.221974 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:11:45.223988 (+ 2014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:11:45.223997 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:11:45.224368 (+ 371us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:45.224396 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 09:11:45.224584 (+ 188us) client_negotiation.cc:240] Negotiation successful
0811 09:11:45.224745 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11637,"mutex_wait_us":53,"thread_start_us":11401,"threads_started":1}
I20260811 09:11:45.225788 26583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:45.204489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36344 (local address 127.28.154.130:37427)
0811 09:11:45.217557 (+ 13068us) server_negotiation.cc:206] Beginning negotiation
0811 09:11:45.217565 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:11:45.217596 (+ 31us) server_negotiation.cc:407] Connection header received
0811 09:11:45.217706 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:11:45.217714 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:11:45.217932 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:11:45.218152 (+ 220us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:11:45.218868 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:45.220442 (+ 1574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:45.222696 (+ 2254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:45.223750 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:45.224674 (+ 924us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:45.224838 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0811 09:11:45.224905 (+ 67us) server_negotiation.cc:299] Negotiation successful
0811 09:11:45.225076 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12902,"thread_start_us":14949,"threads_started":1}
I20260811 09:11:45.226377 26584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:45.204691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41928 (local address 127.28.154.131:38565)
0811 09:11:45.215566 (+ 10875us) server_negotiation.cc:206] Beginning negotiation
0811 09:11:45.215578 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 09:11:45.216332 (+ 754us) server_negotiation.cc:407] Connection header received
0811 09:11:45.216587 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:11:45.216597 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:11:45.216838 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:11:45.217036 (+ 198us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:11:45.217694 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:45.219118 (+ 1424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:45.222163 (+ 3045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:45.223731 (+ 1568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:45.223930 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:45.224119 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0811 09:11:45.224864 (+ 745us) server_negotiation.cc:299] Negotiation successful
0811 09:11:45.225047 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10649,"thread_start_us":10486,"threads_started":1}
I20260811 09:11:45.227090 26585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:45.206223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.129:41979 (local address 127.0.0.1:41406)
0811 09:11:45.218835 (+ 12612us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:45.218854 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 09:11:45.218957 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:11:45.219969 (+ 1012us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:11:45.219973 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:11:45.219997 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:11:45.220468 (+ 471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:11:45.220479 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:11:45.222463 (+ 1984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:11:45.222473 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:11:45.224490 (+ 2017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:11:45.224503 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:11:45.226371 (+ 1868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:11:45.226377 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:11:45.226556 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:45.226584 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 09:11:45.226654 (+ 70us) client_negotiation.cc:240] Negotiation successful
0811 09:11:45.226787 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12490,"thread_start_us":12322,"threads_started":1}
I20260811 09:11:45.227298 26586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:45.206885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41406 (local address 127.28.154.129:41979)
0811 09:11:45.219483 (+ 12598us) server_negotiation.cc:206] Beginning negotiation
0811 09:11:45.219491 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:11:45.219518 (+ 27us) server_negotiation.cc:407] Connection header received
0811 09:11:45.219612 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:11:45.219620 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:11:45.219787 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:11:45.219949 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:11:45.220710 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:45.222271 (+ 1561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:45.224679 (+ 2408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:45.226160 (+ 1481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:45.226348 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:45.226508 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0811 09:11:45.226822 (+ 314us) server_negotiation.cc:299] Negotiation successful
0811 09:11:45.226988 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12444,"thread_start_us":12315,"threads_started":1}
I20260811 09:11:45.230355 26145 tablet_service.cc:1467] Processing CreateTablet for tablet d83be88c5e0f421d9ff10ad93971e1c6 (DEFAULT_TABLE table=test-table [id=8aedb1fed4c34aec9e679945dd964166]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:11:45.230288 26428 tablet_service.cc:1467] Processing CreateTablet for tablet d83be88c5e0f421d9ff10ad93971e1c6 (DEFAULT_TABLE table=test-table [id=8aedb1fed4c34aec9e679945dd964166]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:11:45.230757 25893 tablet_service.cc:1467] Processing CreateTablet for tablet d83be88c5e0f421d9ff10ad93971e1c6 (DEFAULT_TABLE table=test-table [id=8aedb1fed4c34aec9e679945dd964166]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:11:45.231992 26145 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d83be88c5e0f421d9ff10ad93971e1c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:11:45.232023 26428 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d83be88c5e0f421d9ff10ad93971e1c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:11:45.232185 25893 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d83be88c5e0f421d9ff10ad93971e1c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:11:45.244175 26589 tablet_bootstrap.cc:492] T d83be88c5e0f421d9ff10ad93971e1c6 P 82c897eb04564e7d8870a909b3082d02: Bootstrap starting.
I20260811 09:11:45.246779 26592 tablet_bootstrap.cc:492] T d83be88c5e0f421d9ff10ad93971e1c6 P 153e73b493bf47509fd161e335705439: Bootstrap starting.
I20260811 09:11:45.247587 26590 tablet_bootstrap.cc:492] T d83be88c5e0f421d9ff10ad93971e1c6 P 84334312b3254576991f3f49ecc134b1: Bootstrap starting.
I20260811 09:11:45.248042 26589 tablet_bootstrap.cc:654] T d83be88c5e0f421d9ff10ad93971e1c6 P 82c897eb04564e7d8870a909b3082d02: Neither blocks nor log segments found. Creating new log.
I20260811 09:11:45.249470 26589 log.cc:824] T d83be88c5e0f421d9ff10ad93971e1c6 P 82c897eb04564e7d8870a909b3082d02: Log is configured to *not* fsync() on all Append() calls
I20260811 09:11:45.251068 26592 tablet_bootstrap.cc:654] T d83be88c5e0f421d9ff10ad93971e1c6 P 153e73b493bf47509fd161e335705439: Neither blocks nor log segments found. Creating new log.
I20260811 09:11:45.251310 26590 tablet_bootstrap.cc:654] T d83be88c5e0f421d9ff10ad93971e1c6 P 84334312b3254576991f3f49ecc134b1: Neither blocks nor log segments found. Creating new log.
I20260811 09:11:45.252131 26592 log.cc:824] T d83be88c5e0f421d9ff10ad93971e1c6 P 153e73b493bf47509fd161e335705439: Log is configured to *not* fsync() on all Append() calls
I20260811 09:11:45.252622 26590 log.cc:824] T d83be88c5e0f421d9ff10ad93971e1c6 P 84334312b3254576991f3f49ecc134b1: Log is configured to *not* fsync() on all Append() calls
I20260811 09:11:45.255111 26592 tablet_bootstrap.cc:492] T d83be88c5e0f421d9ff10ad93971e1c6 P 153e73b493bf47509fd161e335705439: No bootstrap required, opened a new log
I20260811 09:11:45.255375 26592 ts_tablet_manager.cc:1397] T d83be88c5e0f421d9ff10ad93971e1c6 P 153e73b493bf47509fd161e335705439: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.000s
I20260811 09:11:45.255409 26589 tablet_bootstrap.cc:492] T d83be88c5e0f421d9ff10ad93971e1c6 P 82c897eb04564e7d8870a909b3082d02: No bootstrap required, opened a new log
I20260811 09:11:45.255595 26590 tablet_bootstrap.cc:492] T d83be88c5e0f421d9ff10ad93971e1c6 P 84334312b3254576991f3f49ecc134b1: No bootstrap required, opened a new log
I20260811 09:11:45.255697 26589 ts_tablet_manager.cc:1397] T d83be88c5e0f421d9ff10ad93971e1c6 P 82c897eb04564e7d8870a909b3082d02: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.008s
I20260811 09:11:45.255846 26590 ts_tablet_manager.cc:1397] T d83be88c5e0f421d9ff10ad93971e1c6 P 84334312b3254576991f3f49ecc134b1: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260811 09:11:45.264317 26592 raft_consensus.cc:348] T d83be88c5e0f421d9ff10ad93971e1c6 P 153e73b493bf47509fd161e335705439 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84334312b3254576991f3f49ecc134b1" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 37427 } } peers { permanent_uuid: "82c897eb04564e7d8870a909b3082d02" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 38565 } } peers { permanent_uuid: "153e73b493bf47509fd161e335705439" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41979 } }
I20260811 09:11:45.264806 26592 raft_consensus.cc:374] T d83be88c5e0f421d9ff10ad93971e1c6 P 153e73b493bf47509fd161e335705439 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:11:45.264988 26592 raft_consensus.cc:729] T d83be88c5e0f421d9ff10ad93971e1c6 P 153e73b493bf47509fd161e335705439 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 153e73b493bf47509fd161e335705439, State: Initialized, Role: FOLLOWER
I20260811 09:11:45.265717 26592 consensus_queue.cc:260] T d83be88c5e0f421d9ff10ad93971e1c6 P 153e73b493bf47509fd161e335705439 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84334312b3254576991f3f49ecc134b1" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 37427 } } peers { permanent_uuid: "82c897eb04564e7d8870a909b3082d02" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 38565 } } peers { permanent_uuid: "153e73b493bf47509fd161e335705439" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41979 } }
I20260811 09:11:45.267141 26589 raft_consensus.cc:348] T d83be88c5e0f421d9ff10ad93971e1c6 P 82c897eb04564e7d8870a909b3082d02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84334312b3254576991f3f49ecc134b1" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 37427 } } peers { permanent_uuid: "82c897eb04564e7d8870a909b3082d02" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 38565 } } peers { permanent_uuid: "153e73b493bf47509fd161e335705439" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41979 } }
I20260811 09:11:45.267081 26590 raft_consensus.cc:348] T d83be88c5e0f421d9ff10ad93971e1c6 P 84334312b3254576991f3f49ecc134b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84334312b3254576991f3f49ecc134b1" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 37427 } } peers { permanent_uuid: "82c897eb04564e7d8870a909b3082d02" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 38565 } } peers { permanent_uuid: "153e73b493bf47509fd161e335705439" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41979 } }
I20260811 09:11:45.267699 26590 raft_consensus.cc:374] T d83be88c5e0f421d9ff10ad93971e1c6 P 84334312b3254576991f3f49ecc134b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:11:45.267720 26589 raft_consensus.cc:374] T d83be88c5e0f421d9ff10ad93971e1c6 P 82c897eb04564e7d8870a909b3082d02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:11:45.267841 26592 ts_tablet_manager.cc:1428] T d83be88c5e0f421d9ff10ad93971e1c6 P 153e73b493bf47509fd161e335705439: Time spent starting tablet: real 0.012s user 0.014s sys 0.000s
I20260811 09:11:45.267903 26589 raft_consensus.cc:729] T d83be88c5e0f421d9ff10ad93971e1c6 P 82c897eb04564e7d8870a909b3082d02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82c897eb04564e7d8870a909b3082d02, State: Initialized, Role: FOLLOWER
I20260811 09:11:45.267978 26590 raft_consensus.cc:729] T d83be88c5e0f421d9ff10ad93971e1c6 P 84334312b3254576991f3f49ecc134b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84334312b3254576991f3f49ecc134b1, State: Initialized, Role: FOLLOWER
I20260811 09:11:45.268764 26589 consensus_queue.cc:260] T d83be88c5e0f421d9ff10ad93971e1c6 P 82c897eb04564e7d8870a909b3082d02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84334312b3254576991f3f49ecc134b1" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 37427 } } peers { permanent_uuid: "82c897eb04564e7d8870a909b3082d02" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 38565 } } peers { permanent_uuid: "153e73b493bf47509fd161e335705439" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41979 } }
I20260811 09:11:45.268765 26590 consensus_queue.cc:260] T d83be88c5e0f421d9ff10ad93971e1c6 P 84334312b3254576991f3f49ecc134b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84334312b3254576991f3f49ecc134b1" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 37427 } } peers { permanent_uuid: "82c897eb04564e7d8870a909b3082d02" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 38565 } } peers { permanent_uuid: "153e73b493bf47509fd161e335705439" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41979 } }
I20260811 09:11:45.270367 26286 heartbeater.cc:503] Master 127.28.154.190:40403 was elected leader, sending a full tablet report...
I20260811 09:11:45.270414 26558 heartbeater.cc:503] Master 127.28.154.190:40403 was elected leader, sending a full tablet report...
I20260811 09:11:45.270987 26590 ts_tablet_manager.cc:1428] T d83be88c5e0f421d9ff10ad93971e1c6 P 84334312b3254576991f3f49ecc134b1: Time spent starting tablet: real 0.015s user 0.010s sys 0.006s
I20260811 09:11:45.272167 26589 ts_tablet_manager.cc:1428] T d83be88c5e0f421d9ff10ad93971e1c6 P 82c897eb04564e7d8870a909b3082d02: Time spent starting tablet: real 0.016s user 0.011s sys 0.005s
W20260811 09:11:45.282052 26023 tablet.cc:2369] T d83be88c5e0f421d9ff10ad93971e1c6 P 153e73b493bf47509fd161e335705439: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 09:11:45.330749 26287 tablet.cc:2369] T d83be88c5e0f421d9ff10ad93971e1c6 P 84334312b3254576991f3f49ecc134b1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 09:11:45.381392 26559 tablet.cc:2369] T d83be88c5e0f421d9ff10ad93971e1c6 P 82c897eb04564e7d8870a909b3082d02: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 09:11:45.473634 26602 raft_consensus.cc:482] T d83be88c5e0f421d9ff10ad93971e1c6 P 84334312b3254576991f3f49ecc134b1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:11:45.473977 26602 raft_consensus.cc:504] T d83be88c5e0f421d9ff10ad93971e1c6 P 84334312b3254576991f3f49ecc134b1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84334312b3254576991f3f49ecc134b1" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 37427 } } peers { permanent_uuid: "82c897eb04564e7d8870a909b3082d02" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 38565 } } peers { permanent_uuid: "153e73b493bf47509fd161e335705439" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41979 } }
I20260811 09:11:45.475734 26602 leader_election.cc:288] T d83be88c5e0f421d9ff10ad93971e1c6 P 84334312b3254576991f3f49ecc134b1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 82c897eb04564e7d8870a909b3082d02 (127.28.154.131:38565), 153e73b493bf47509fd161e335705439 (127.28.154.129:41979)
I20260811 09:11:45.483541 26600 raft_consensus.cc:482] T d83be88c5e0f421d9ff10ad93971e1c6 P 82c897eb04564e7d8870a909b3082d02 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:11:45.483899 26600 raft_consensus.cc:504] T d83be88c5e0f421d9ff10ad93971e1c6 P 82c897eb04564e7d8870a909b3082d02 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84334312b3254576991f3f49ecc134b1" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 37427 } } peers { permanent_uuid: "82c897eb04564e7d8870a909b3082d02" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 38565 } } peers { permanent_uuid: "153e73b493bf47509fd161e335705439" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41979 } }
I20260811 09:11:45.486311 26600 leader_election.cc:288] T d83be88c5e0f421d9ff10ad93971e1c6 P 82c897eb04564e7d8870a909b3082d02 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 84334312b3254576991f3f49ecc134b1 (127.28.154.130:37427), 153e73b493bf47509fd161e335705439 (127.28.154.129:41979)
I20260811 09:11:45.486519 26584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:45.476474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.130:33665 (local address 127.28.154.131:38565)
0811 09:11:45.476686 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0811 09:11:45.476792 (+ 106us) server_negotiation.cc:399] Waiting for connection header
0811 09:11:45.477565 (+ 773us) server_negotiation.cc:407] Connection header received
0811 09:11:45.477763 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:11:45.477770 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:11:45.477855 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:11:45.477964 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:11:45.479452 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:45.480659 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:45.484134 (+ 3475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:45.485493 (+ 1359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:45.485698 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:45.485776 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 09:11:45.486065 (+ 289us) server_negotiation.cc:299] Negotiation successful
0811 09:11:45.486201 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 09:11:45.486639 26604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:45.476400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.131:38565 (local address 127.28.154.130:33665)
0811 09:11:45.477448 (+ 1048us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:45.477463 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 09:11:45.477561 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:11:45.478156 (+ 595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:11:45.478164 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:11:45.478249 (+ 85us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:11:45.479212 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:11:45.479232 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:11:45.480922 (+ 1690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:11:45.480928 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:11:45.483894 (+ 2966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:11:45.483909 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:11:45.485738 (+ 1829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:11:45.485744 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:11:45.485871 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:45.485899 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 09:11:45.485999 (+ 100us) client_negotiation.cc:240] Negotiation successful
0811 09:11:45.486147 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":929,"mutex_wait_us":343,"spinlock_wait_cycles":47616,"thread_start_us":542,"threads_started":1}
I20260811 09:11:45.488466 26586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:45.476387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.130:48753 (local address 127.28.154.129:41979)
0811 09:11:45.476605 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0811 09:11:45.476611 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:11:45.477509 (+ 898us) server_negotiation.cc:407] Connection header received
0811 09:11:45.477763 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:11:45.477770 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:11:45.477868 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:11:45.477964 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:11:45.479453 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:45.480660 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:45.485902 (+ 5242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:45.487244 (+ 1342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:45.487443 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:45.487517 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 09:11:45.488036 (+ 519us) server_negotiation.cc:299] Negotiation successful
0811 09:11:45.488183 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 09:11:45.489477 26607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:45.476366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.129:41979 (local address 127.28.154.130:48753)
0811 09:11:45.477336 (+ 970us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:45.477360 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 09:11:45.477559 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:11:45.478156 (+ 597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:11:45.478164 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:11:45.478258 (+ 94us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:11:45.479221 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:11:45.479231 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:11:45.480922 (+ 1691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:11:45.480928 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:11:45.483887 (+ 2959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:11:45.483909 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:11:45.487504 (+ 3595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:11:45.487511 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:11:45.487787 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:45.487825 (+ 38us) client_negotiation.cc:769] Sending connection context
0811 09:11:45.488911 (+ 1086us) client_negotiation.cc:240] Negotiation successful
0811 09:11:45.489074 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":807,"mutex_wait_us":189,"spinlock_wait_cycles":59136,"thread_start_us":594,"threads_started":1}
I20260811 09:11:45.489612 26466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d83be88c5e0f421d9ff10ad93971e1c6" candidate_uuid: "84334312b3254576991f3f49ecc134b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "82c897eb04564e7d8870a909b3082d02" is_pre_election: true
I20260811 09:11:45.490181 26466 raft_consensus.cc:2454] T d83be88c5e0f421d9ff10ad93971e1c6 P 82c897eb04564e7d8870a909b3082d02 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84334312b3254576991f3f49ecc134b1 in term 0.
I20260811 09:11:45.491189 26066 leader_election.cc:302] T d83be88c5e0f421d9ff10ad93971e1c6 P 84334312b3254576991f3f49ecc134b1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 82c897eb04564e7d8870a909b3082d02, 84334312b3254576991f3f49ecc134b1; no voters:
I20260811 09:11:45.491304 25927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d83be88c5e0f421d9ff10ad93971e1c6" candidate_uuid: "84334312b3254576991f3f49ecc134b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "153e73b493bf47509fd161e335705439" is_pre_election: true
I20260811 09:11:45.491935 25927 raft_consensus.cc:2454] T d83be88c5e0f421d9ff10ad93971e1c6 P 153e73b493bf47509fd161e335705439 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 84334312b3254576991f3f49ecc134b1 in term 0.
I20260811 09:11:45.492043 26602 raft_consensus.cc:2781] T d83be88c5e0f421d9ff10ad93971e1c6 P 84334312b3254576991f3f49ecc134b1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 09:11:45.492213 26602 raft_consensus.cc:482] T d83be88c5e0f421d9ff10ad93971e1c6 P 84334312b3254576991f3f49ecc134b1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 09:11:45.492312 26602 raft_consensus.cc:3037] T d83be88c5e0f421d9ff10ad93971e1c6 P 84334312b3254576991f3f49ecc134b1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:11:45.494310 26561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:45.486974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.130:37427 (local address 127.28.154.131:48415)
0811 09:11:45.487177 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:45.487195 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 09:11:45.487295 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:11:45.488148 (+ 853us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:11:45.488154 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:11:45.488184 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:11:45.488403 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:11:45.488415 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:11:45.489822 (+ 1407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:11:45.489828 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:11:45.491753 (+ 1925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:11:45.491769 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:11:45.493534 (+ 1765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:11:45.493540 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:11:45.493767 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:45.493807 (+ 40us) client_negotiation.cc:769] Sending connection context
0811 09:11:45.493898 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 09:11:45.494051 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":76}
I20260811 09:11:45.494445 26583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:45.487681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.131:48415 (local address 127.28.154.130:37427)
0811 09:11:45.487867 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 09:11:45.487874 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:11:45.487894 (+ 20us) server_negotiation.cc:407] Connection header received
0811 09:11:45.487959 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:11:45.487963 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:11:45.488030 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:11:45.488109 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:11:45.488621 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:45.489639 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:45.491911 (+ 2272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:45.493212 (+ 1301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:45.493368 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:45.493439 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 09:11:45.494041 (+ 602us) server_negotiation.cc:299] Negotiation successful
0811 09:11:45.494188 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 09:11:45.495288 26186 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d83be88c5e0f421d9ff10ad93971e1c6" candidate_uuid: "82c897eb04564e7d8870a909b3082d02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84334312b3254576991f3f49ecc134b1" is_pre_election: true
I20260811 09:11:45.495697 26602 raft_consensus.cc:504] T d83be88c5e0f421d9ff10ad93971e1c6 P 84334312b3254576991f3f49ecc134b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84334312b3254576991f3f49ecc134b1" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 37427 } } peers { permanent_uuid: "82c897eb04564e7d8870a909b3082d02" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 38565 } } peers { permanent_uuid: "153e73b493bf47509fd161e335705439" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41979 } }
I20260811 09:11:45.496158 26186 raft_consensus.cc:2379] T d83be88c5e0f421d9ff10ad93971e1c6 P 84334312b3254576991f3f49ecc134b1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 82c897eb04564e7d8870a909b3082d02 in current term 1: Already voted for candidate 84334312b3254576991f3f49ecc134b1 in this term.
I20260811 09:11:45.496511 26609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:45.487529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.129:41979 (local address 127.28.154.131:55605)
0811 09:11:45.488903 (+ 1374us) negotiation.cc:107] Waiting for socket to connect
0811 09:11:45.488928 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 09:11:45.489100 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:11:45.490111 (+ 1011us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:11:45.490115 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:11:45.490141 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:11:45.490755 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:11:45.490769 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:11:45.492307 (+ 1538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:11:45.492311 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:11:45.493846 (+ 1535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:11:45.493859 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:11:45.495661 (+ 1802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:11:45.495667 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:11:45.495845 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:45.495870 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 09:11:45.495966 (+ 96us) client_negotiation.cc:240] Negotiation successful
0811 09:11:45.496101 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1251,"thread_start_us":1095,"threads_started":1}
I20260811 09:11:45.496647 26586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:45.487787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.131:55605 (local address 127.28.154.129:41979)
0811 09:11:45.488868 (+ 1081us) server_negotiation.cc:206] Beginning negotiation
0811 09:11:45.488874 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:11:45.489005 (+ 131us) server_negotiation.cc:407] Connection header received
0811 09:11:45.489765 (+ 760us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:11:45.489771 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:11:45.489846 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:11:45.489918 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:11:45.490975 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:45.492111 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:45.494013 (+ 1902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:45.495467 (+ 1454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:45.495623 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:45.495691 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 09:11:45.496000 (+ 309us) server_negotiation.cc:299] Negotiation successful
0811 09:11:45.496129 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":979,"thread_start_us":1693,"threads_started":1}
I20260811 09:11:45.497042 26466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d83be88c5e0f421d9ff10ad93971e1c6" candidate_uuid: "84334312b3254576991f3f49ecc134b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "82c897eb04564e7d8870a909b3082d02"
I20260811 09:11:45.497283 26466 raft_consensus.cc:3037] T d83be88c5e0f421d9ff10ad93971e1c6 P 82c897eb04564e7d8870a909b3082d02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:11:45.497344 26602 leader_election.cc:288] T d83be88c5e0f421d9ff10ad93971e1c6 P 84334312b3254576991f3f49ecc134b1 [CANDIDATE]: Term 1 election: Requested vote from peers 82c897eb04564e7d8870a909b3082d02 (127.28.154.131:38565), 153e73b493bf47509fd161e335705439 (127.28.154.129:41979)
I20260811 09:11:45.497329 25927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d83be88c5e0f421d9ff10ad93971e1c6" candidate_uuid: "82c897eb04564e7d8870a909b3082d02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "153e73b493bf47509fd161e335705439" is_pre_election: true
I20260811 09:11:45.497686 25927 raft_consensus.cc:2454] T d83be88c5e0f421d9ff10ad93971e1c6 P 153e73b493bf47509fd161e335705439 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 82c897eb04564e7d8870a909b3082d02 in term 0.
I20260811 09:11:45.498034 25924 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d83be88c5e0f421d9ff10ad93971e1c6" candidate_uuid: "84334312b3254576991f3f49ecc134b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "153e73b493bf47509fd161e335705439"
I20260811 09:11:45.498310 25924 raft_consensus.cc:3037] T d83be88c5e0f421d9ff10ad93971e1c6 P 153e73b493bf47509fd161e335705439 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:11:45.498234 26330 leader_election.cc:302] T d83be88c5e0f421d9ff10ad93971e1c6 P 82c897eb04564e7d8870a909b3082d02 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 153e73b493bf47509fd161e335705439, 82c897eb04564e7d8870a909b3082d02; no voters: 84334312b3254576991f3f49ecc134b1
I20260811 09:11:45.500252 26466 raft_consensus.cc:2454] T d83be88c5e0f421d9ff10ad93971e1c6 P 82c897eb04564e7d8870a909b3082d02 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84334312b3254576991f3f49ecc134b1 in term 1.
I20260811 09:11:45.500488 26600 raft_consensus.cc:2741] T d83be88c5e0f421d9ff10ad93971e1c6 P 82c897eb04564e7d8870a909b3082d02 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260811 09:11:45.500798 26066 leader_election.cc:302] T d83be88c5e0f421d9ff10ad93971e1c6 P 84334312b3254576991f3f49ecc134b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 82c897eb04564e7d8870a909b3082d02, 84334312b3254576991f3f49ecc134b1; no voters:
I20260811 09:11:45.501194 26602 raft_consensus.cc:2781] T d83be88c5e0f421d9ff10ad93971e1c6 P 84334312b3254576991f3f49ecc134b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:11:45.501359 25924 raft_consensus.cc:2454] T d83be88c5e0f421d9ff10ad93971e1c6 P 153e73b493bf47509fd161e335705439 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 84334312b3254576991f3f49ecc134b1 in term 1.
I20260811 09:11:45.501734 26602 raft_consensus.cc:686] T d83be88c5e0f421d9ff10ad93971e1c6 P 84334312b3254576991f3f49ecc134b1 [term 1 LEADER]: Becoming Leader. State: Replica: 84334312b3254576991f3f49ecc134b1, State: Running, Role: LEADER
I20260811 09:11:45.502313 26602 consensus_queue.cc:237] T d83be88c5e0f421d9ff10ad93971e1c6 P 84334312b3254576991f3f49ecc134b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84334312b3254576991f3f49ecc134b1" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 37427 } } peers { permanent_uuid: "82c897eb04564e7d8870a909b3082d02" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 38565 } } peers { permanent_uuid: "153e73b493bf47509fd161e335705439" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41979 } }
I20260811 09:11:45.507184 25656 catalog_manager.cc:5495] T d83be88c5e0f421d9ff10ad93971e1c6 P 84334312b3254576991f3f49ecc134b1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 84334312b3254576991f3f49ecc134b1 (127.28.154.130). New cstate: current_term: 1 leader_uuid: "84334312b3254576991f3f49ecc134b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "84334312b3254576991f3f49ecc134b1" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 37427 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "82c897eb04564e7d8870a909b3082d02" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 38565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "153e73b493bf47509fd161e335705439" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41979 } health_report { overall_health: UNKNOWN } } }
I20260811 09:11:45.531142 26583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:45.525388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36356 (local address 127.28.154.130:37427)
0811 09:11:45.525622 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0811 09:11:45.525629 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:11:45.525710 (+ 81us) server_negotiation.cc:407] Connection header received
0811 09:11:45.525957 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:11:45.525961 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:11:45.526056 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:11:45.526190 (+ 134us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:11:45.526615 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:45.527368 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:45.528927 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:45.529334 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:45.529450 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:45.529966 (+ 516us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:11:45.530515 (+ 549us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:11:45.530650 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0811 09:11:45.530759 (+ 109us) server_negotiation.cc:299] Negotiation successful
0811 09:11:45.530848 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109}
I20260811 09:11:45.539736 26466 raft_consensus.cc:1261] T d83be88c5e0f421d9ff10ad93971e1c6 P 82c897eb04564e7d8870a909b3082d02 [term 1 FOLLOWER]: Refusing update from remote peer 84334312b3254576991f3f49ecc134b1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:11:45.539863 25924 raft_consensus.cc:1261] T d83be88c5e0f421d9ff10ad93971e1c6 P 153e73b493bf47509fd161e335705439 [term 1 FOLLOWER]: Refusing update from remote peer 84334312b3254576991f3f49ecc134b1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:11:45.540901 26602 consensus_queue.cc:1035] T d83be88c5e0f421d9ff10ad93971e1c6 P 84334312b3254576991f3f49ecc134b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "82c897eb04564e7d8870a909b3082d02" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 38565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:11:45.541217 26613 consensus_queue.cc:1035] T d83be88c5e0f421d9ff10ad93971e1c6 P 84334312b3254576991f3f49ecc134b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "153e73b493bf47509fd161e335705439" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:11:45.551473 26622 mvcc.cc:205] Tried to move back new op lower bound from 7317256214679633920 to 7317256214541717504. Current Snapshot: MvccSnapshot[applied={T|T < 7317256214679633920}]
I20260811 09:11:45.566017 25656 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41144:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:11:45.566377 25656 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41144:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:11:45.570441 25656 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b740e22f6c334fb38cdc579e01606268: Sending DeleteTablet for 3 replicas of tablet d83be88c5e0f421d9ff10ad93971e1c6
I20260811 09:11:45.571600 26428 tablet_service.cc:1514] Processing DeleteTablet for tablet d83be88c5e0f421d9ff10ad93971e1c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:11:45 UTC) from {username='kudu', principal='kudu/127.28.154.190@KRBTEST.COM'} at 127.0.0.1:41928
I20260811 09:11:45.571903 26145 tablet_service.cc:1514] Processing DeleteTablet for tablet d83be88c5e0f421d9ff10ad93971e1c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:11:45 UTC) from {username='kudu', principal='kudu/127.28.154.190@KRBTEST.COM'} at 127.0.0.1:36344
I20260811 09:11:45.572018 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 25778
I20260811 09:11:45.573271 25893 tablet_service.cc:1514] Processing DeleteTablet for tablet d83be88c5e0f421d9ff10ad93971e1c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:11:45 UTC) from {username='kudu', principal='kudu/127.28.154.190@KRBTEST.COM'} at 127.0.0.1:41406
I20260811 09:11:45.574383 25892 tablet_service.cc:1514] Processing DeleteTablet for tablet d83be88c5e0f421d9ff10ad93971e1c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:11:45 UTC) from {username='kudu', principal='kudu/127.28.154.190@KRBTEST.COM'} at 127.0.0.1:41406
I20260811 09:11:45.584271 26638 tablet_replica.cc:330] T d83be88c5e0f421d9ff10ad93971e1c6 P 82c897eb04564e7d8870a909b3082d02: stopping tablet replica
I20260811 09:11:45.584265 26644 tablet_replica.cc:330] T d83be88c5e0f421d9ff10ad93971e1c6 P 84334312b3254576991f3f49ecc134b1: stopping tablet replica
I20260811 09:11:45.584761 26644 raft_consensus.cc:2229] T d83be88c5e0f421d9ff10ad93971e1c6 P 84334312b3254576991f3f49ecc134b1 [term 1 LEADER]: Raft consensus shutting down.
I20260811 09:11:45.584872 26638 raft_consensus.cc:2229] T d83be88c5e0f421d9ff10ad93971e1c6 P 82c897eb04564e7d8870a909b3082d02 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:11:45.585446 26638 raft_consensus.cc:2258] T d83be88c5e0f421d9ff10ad93971e1c6 P 82c897eb04564e7d8870a909b3082d02 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:11:45.585582 26644 raft_consensus.cc:2258] T d83be88c5e0f421d9ff10ad93971e1c6 P 84334312b3254576991f3f49ecc134b1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:11:45.587877 26644 ts_tablet_manager.cc:1905] T d83be88c5e0f421d9ff10ad93971e1c6 P 84334312b3254576991f3f49ecc134b1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:11:45.587869 26638 ts_tablet_manager.cc:1905] T d83be88c5e0f421d9ff10ad93971e1c6 P 82c897eb04564e7d8870a909b3082d02: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:11:45.594971 26644 ts_tablet_manager.cc:1918] T d83be88c5e0f421d9ff10ad93971e1c6 P 84334312b3254576991f3f49ecc134b1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:11:45.594966 26638 ts_tablet_manager.cc:1918] T d83be88c5e0f421d9ff10ad93971e1c6 P 82c897eb04564e7d8870a909b3082d02: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:11:45.595150 26638 log.cc:1196] T d83be88c5e0f421d9ff10ad93971e1c6 P 82c897eb04564e7d8870a909b3082d02: Deleting WAL directory at /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-2/wal/wals/d83be88c5e0f421d9ff10ad93971e1c6
I20260811 09:11:45.595264 26644 log.cc:1196] T d83be88c5e0f421d9ff10ad93971e1c6 P 84334312b3254576991f3f49ecc134b1: Deleting WAL directory at /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-1/wal/wals/d83be88c5e0f421d9ff10ad93971e1c6
I20260811 09:11:45.595773 26638 ts_tablet_manager.cc:1939] T d83be88c5e0f421d9ff10ad93971e1c6 P 82c897eb04564e7d8870a909b3082d02: Deleting consensus metadata
I20260811 09:11:45.595853 26644 ts_tablet_manager.cc:1939] T d83be88c5e0f421d9ff10ad93971e1c6 P 84334312b3254576991f3f49ecc134b1: Deleting consensus metadata
I20260811 09:11:45.597927 25632 catalog_manager.cc:4841] TS 82c897eb04564e7d8870a909b3082d02 (127.28.154.131:38565): tablet d83be88c5e0f421d9ff10ad93971e1c6 (table test-table [id=8aedb1fed4c34aec9e679945dd964166]) successfully deleted
I20260811 09:11:45.598235 25629 catalog_manager.cc:4841] TS 84334312b3254576991f3f49ecc134b1 (127.28.154.130:37427): tablet d83be88c5e0f421d9ff10ad93971e1c6 (table test-table [id=8aedb1fed4c34aec9e679945dd964166]) successfully deleted
I20260811 09:11:45.601495 26646 tablet_replica.cc:330] T d83be88c5e0f421d9ff10ad93971e1c6 P 153e73b493bf47509fd161e335705439: stopping tablet replica
I20260811 09:11:45.601975 26646 raft_consensus.cc:2229] T d83be88c5e0f421d9ff10ad93971e1c6 P 153e73b493bf47509fd161e335705439 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:11:45.602563 26646 raft_consensus.cc:2258] T d83be88c5e0f421d9ff10ad93971e1c6 P 153e73b493bf47509fd161e335705439 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:11:45.605379 26646 ts_tablet_manager.cc:1905] T d83be88c5e0f421d9ff10ad93971e1c6 P 153e73b493bf47509fd161e335705439: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:11:45.613061 26646 ts_tablet_manager.cc:1918] T d83be88c5e0f421d9ff10ad93971e1c6 P 153e73b493bf47509fd161e335705439: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:11:45.613332 26646 log.cc:1196] T d83be88c5e0f421d9ff10ad93971e1c6 P 153e73b493bf47509fd161e335705439: Deleting WAL directory at /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-0/wal/wals/d83be88c5e0f421d9ff10ad93971e1c6
I20260811 09:11:45.613905 26646 ts_tablet_manager.cc:1939] T d83be88c5e0f421d9ff10ad93971e1c6 P 153e73b493bf47509fd161e335705439: Deleting consensus metadata
Aug 11 09:11:45 dist-test-slave-xt4k krb5kdc[25585](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439502, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.154.129@KRBTEST.COM
I20260811 09:11:45.616087 25629 catalog_manager.cc:4841] TS 153e73b493bf47509fd161e335705439 (127.28.154.129:41979): tablet d83be88c5e0f421d9ff10ad93971e1c6 (table test-table [id=8aedb1fed4c34aec9e679945dd964166]) successfully deleted
W20260811 09:11:45.617396 25629 catalog_manager.cc:4808] TS 153e73b493bf47509fd161e335705439 (127.28.154.129:41979): delete failed for tablet d83be88c5e0f421d9ff10ad93971e1c6 because the tablet was not found. No further retry: Not found: Tablet not found: d83be88c5e0f421d9ff10ad93971e1c6
I20260811 09:11:45.623680 26586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:45.574118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41414 (local address 127.28.154.129:41979)
0811 09:11:45.574311 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 09:11:45.574318 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:11:45.603772 (+ 29454us) server_negotiation.cc:407] Connection header received
0811 09:11:45.603960 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:11:45.603967 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:11:45.604077 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:11:45.604185 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:11:45.606387 (+ 2202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:45.607562 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:45.609686 (+ 2124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:45.610398 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:45.610519 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:45.616434 (+ 5915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:11:45.616449 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:11:45.616468 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:11:45.616512 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:11:45.620456 (+ 3944us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:11:45.621370 (+ 914us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:11:45.621381 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:11:45.621385 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:11:45.621554 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:11:45.621922 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:11:45.621927 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:11:45.621931 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:11:45.622419 (+ 488us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:11:45.622731 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0811 09:11:45.623074 (+ 343us) server_negotiation.cc:299] Negotiation successful
0811 09:11:45.623308 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 09:11:45.624501 26012 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:11:45.741714 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 25778
I20260811 09:11:45.771956 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 26031
Aug 11 09:11:45 dist-test-slave-xt4k krb5kdc[25585](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439502, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.154.130@KRBTEST.COM
I20260811 09:11:45.791441 26583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:45.772799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36360 (local address 127.28.154.130:37427)
0811 09:11:45.772997 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0811 09:11:45.773002 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:11:45.773082 (+ 80us) server_negotiation.cc:407] Connection header received
0811 09:11:45.773271 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:11:45.773275 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:11:45.773367 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:11:45.773460 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:11:45.775318 (+ 1858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:45.776121 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:45.778152 (+ 2031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:45.778646 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:45.778830 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:45.784021 (+ 5191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:11:45.784040 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:11:45.784065 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:11:45.784126 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:11:45.788204 (+ 4078us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:11:45.789000 (+ 796us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:11:45.789010 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:11:45.789014 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:11:45.789205 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:11:45.789667 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:11:45.789672 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:11:45.789676 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:11:45.790220 (+ 544us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:11:45.790564 (+ 344us) server_negotiation.cc:1035] Waiting for connection context
0811 09:11:45.790815 (+ 251us) server_negotiation.cc:299] Negotiation successful
0811 09:11:45.791061 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 09:11:45.792244 26272 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:11:45.912132 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 26031
I20260811 09:11:45.937470 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 26294
Aug 11 09:11:45 dist-test-slave-xt4k krb5kdc[25585](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439502, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.154.131@KRBTEST.COM
I20260811 09:11:45.954574 26584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:11:45.938263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41932 (local address 127.28.154.131:38565)
0811 09:11:45.938478 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0811 09:11:45.938485 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:11:45.938511 (+ 26us) server_negotiation.cc:407] Connection header received
0811 09:11:45.938730 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:11:45.938734 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:11:45.938803 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:11:45.938901 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:11:45.940247 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:45.941065 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:45.942423 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:11:45.942825 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:11:45.942937 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:11:45.948405 (+ 5468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:11:45.948429 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:11:45.948451 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:11:45.948495 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:11:45.951417 (+ 2922us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:11:45.952224 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:11:45.952233 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:11:45.952236 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:11:45.952448 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:11:45.952988 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:11:45.952992 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:11:45.952995 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:11:45.953432 (+ 437us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:11:45.953767 (+ 335us) server_negotiation.cc:1035] Waiting for connection context
0811 09:11:45.954094 (+ 327us) server_negotiation.cc:299] Negotiation successful
0811 09:11:45.954280 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260811 09:11:45.955259 26550 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:11:46.049453 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 26294
I20260811 09:11:46.070979 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 25604
I20260811 09:11:46.072146 25719 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:11:46.184700 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 25604
I20260811 09:12:06.213434 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.154.190:40403
--webserver_interface=127.28.154.190
--webserver_port=44341
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab
--principal=kudu/127.28.154.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:38289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.154.190:40403
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:12:06.585018 26652 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:12:06.585333 26652 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:12:06.585436 26652 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:12:06.592758 26652 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 09:12:06.592868 26652 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 09:12:06.592907 26652 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 09:12:06.592952 26652 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:12:06.593005 26652 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:12:06.593075 26652 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:12:06.593119 26652 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:12:06.593173 26652 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:12:06.603295 26652 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:38289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.28.154.190:40403
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab
--principal=kudu/127.28.154.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.154.190:40403
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.154.190
--webserver_port=44341
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:12:06.604605 26652 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:12:06.605883 26652 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:12:06.631436 26668 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:12:06.631496 26664 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:12:06.631541 26670 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:12:06.632182 26652 server_base.cc:1034] running on GCE node
I20260811 09:12:06.633154 26652 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:12:06.634856 26652 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:12:06.636097 26652 hybrid_clock.cc:648] HybridClock initialized: now 1786439526636026 us; error 39 us; skew 500 ppm
I20260811 09:12:06.636572 26652 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:12:06 dist-test-slave-xt4k krb5kdc[25585](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439526, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:12:06.640467 26652 init.cc:377] Logged in from keytab as kudu/127.28.154.190@KRBTEST.COM (short username kudu)
I20260811 09:12:06.642321 26652 webserver.cc:460] Webserver started at http://127.28.154.190:44341/ using document root <none> and password file <none>
I20260811 09:12:06.643183 26652 fs_manager.cc:359] Metadata directory not provided
I20260811 09:12:06.643324 26652 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:12:06.649664 26652 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.002s
I20260811 09:12:06.652815 26687 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:12:06.654273 26652 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260811 09:12:06.654487 26652 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/master-0/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/master-0/wal
uuid: "b740e22f6c334fb38cdc579e01606268"
format_stamp: "Formatted at 2026-08-11 09:11:43 on dist-test-slave-xt4k"
server_key: "ada7b95799df2469c5c6b10b1a71d5d8"
server_key_iv: "76280d86cd4f431e0dd06f2758862612"
server_key_version: "encryptionkey@0"
I20260811 09:12:06.655591 26652 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:12:06.682267 26652 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:12:06.687132 26652 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:12:06.687443 26652 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:12:06.706830 26652 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.190:40403
I20260811 09:12:06.706882 26773 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.190:40403 every 8 connection(s)
I20260811 09:12:06.708504 26652 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
I20260811 09:12:06.715960 26774 sys_catalog.cc:263] Verifying existing consensus state
I20260811 09:12:06.716434 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 26652
I20260811 09:12:06.717325 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.154.129:41979
--local_ip_for_outbound_sockets=127.28.154.129
--tserver_master_addrs=127.28.154.190:40403
--webserver_port=36297
--webserver_interface=127.28.154.129
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.129.keytab
--principal=kudu/127.28.154.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:38289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 09:12:06.719801 26774 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b740e22f6c334fb38cdc579e01606268: Bootstrap starting.
I20260811 09:12:06.740314 26774 log.cc:824] T 00000000000000000000000000000000 P b740e22f6c334fb38cdc579e01606268: Log is configured to *not* fsync() on all Append() calls
I20260811 09:12:06.754710 26774 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b740e22f6c334fb38cdc579e01606268: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 09:12:06.755314 26774 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b740e22f6c334fb38cdc579e01606268: Bootstrap complete.
I20260811 09:12:06.765354 26774 raft_consensus.cc:348] T 00000000000000000000000000000000 P b740e22f6c334fb38cdc579e01606268 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b740e22f6c334fb38cdc579e01606268" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 40403 } }
I20260811 09:12:06.766258 26774 raft_consensus.cc:729] T 00000000000000000000000000000000 P b740e22f6c334fb38cdc579e01606268 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b740e22f6c334fb38cdc579e01606268, State: Initialized, Role: FOLLOWER
I20260811 09:12:06.766923 26774 consensus_queue.cc:260] T 00000000000000000000000000000000 P b740e22f6c334fb38cdc579e01606268 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b740e22f6c334fb38cdc579e01606268" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 40403 } }
I20260811 09:12:06.767175 26774 raft_consensus.cc:388] T 00000000000000000000000000000000 P b740e22f6c334fb38cdc579e01606268 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:12:06.767297 26774 raft_consensus.cc:482] T 00000000000000000000000000000000 P b740e22f6c334fb38cdc579e01606268 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:12:06.767468 26774 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b740e22f6c334fb38cdc579e01606268 [term 1 FOLLOWER]: Advancing to term 2
I20260811 09:12:06.771628 26774 raft_consensus.cc:504] T 00000000000000000000000000000000 P b740e22f6c334fb38cdc579e01606268 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b740e22f6c334fb38cdc579e01606268" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 40403 } }
I20260811 09:12:06.772373 26774 leader_election.cc:302] T 00000000000000000000000000000000 P b740e22f6c334fb38cdc579e01606268 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b740e22f6c334fb38cdc579e01606268; no voters:
I20260811 09:12:06.773038 26774 leader_election.cc:288] T 00000000000000000000000000000000 P b740e22f6c334fb38cdc579e01606268 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 09:12:06.773193 26780 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b740e22f6c334fb38cdc579e01606268 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 09:12:06.775473 26774 sys_catalog.cc:564] T 00000000000000000000000000000000 P b740e22f6c334fb38cdc579e01606268 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:12:06.780193 26780 raft_consensus.cc:686] T 00000000000000000000000000000000 P b740e22f6c334fb38cdc579e01606268 [term 2 LEADER]: Becoming Leader. State: Replica: b740e22f6c334fb38cdc579e01606268, State: Running, Role: LEADER
I20260811 09:12:06.780938 26780 consensus_queue.cc:237] T 00000000000000000000000000000000 P b740e22f6c334fb38cdc579e01606268 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b740e22f6c334fb38cdc579e01606268" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 40403 } }
I20260811 09:12:06.783828 26781 sys_catalog.cc:455] T 00000000000000000000000000000000 P b740e22f6c334fb38cdc579e01606268 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "b740e22f6c334fb38cdc579e01606268" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b740e22f6c334fb38cdc579e01606268" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 40403 } } }
I20260811 09:12:06.784277 26781 sys_catalog.cc:458] T 00000000000000000000000000000000 P b740e22f6c334fb38cdc579e01606268 [sys.catalog]: This master's current role is: LEADER
I20260811 09:12:06.784849 26783 sys_catalog.cc:455] T 00000000000000000000000000000000 P b740e22f6c334fb38cdc579e01606268 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b740e22f6c334fb38cdc579e01606268. Latest consensus state: current_term: 2 leader_uuid: "b740e22f6c334fb38cdc579e01606268" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b740e22f6c334fb38cdc579e01606268" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 40403 } } }
I20260811 09:12:06.785151 26783 sys_catalog.cc:458] T 00000000000000000000000000000000 P b740e22f6c334fb38cdc579e01606268 [sys.catalog]: This master's current role is: LEADER
I20260811 09:12:06.787267 26797 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:12:06.795348 26797 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:12:06.797040 26797 catalog_manager.cc:1259] Loaded cluster ID: 959c088f5a144412a35c0e95cbab5cb2
I20260811 09:12:06.797240 26797 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:12:06.801748 26797 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:12:06.803503 26797 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b740e22f6c334fb38cdc579e01606268: Loaded TSK: 0
I20260811 09:12:06.833449 26797 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b740e22f6c334fb38cdc579e01606268: Generated new TSK 1
I20260811 09:12:06.833853 26797 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 09:12:07.104286 26777 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:12:07.104668 26777 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:12:07.104808 26777 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:12:07.112210 26777 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:12:07.112377 26777 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:12:07.112582 26777 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.154.129
I20260811 09:12:07.122915 26777 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:38289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.129.keytab
--principal=kudu/127.28.154.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.154.129:41979
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.154.129
--webserver_port=36297
--tserver_master_addrs=127.28.154.190:40403
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.154.129
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:12:07.124300 26777 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:12:07.125833 26777 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:12:07.141706 26823 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:12:07.141887 26824 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:12:07.142939 26830 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:12:07.143486 26777 server_base.cc:1034] running on GCE node
I20260811 09:12:07.144158 26777 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:12:07.145239 26777 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:12:07.146508 26777 hybrid_clock.cc:648] HybridClock initialized: now 1786439527146460 us; error 41 us; skew 500 ppm
I20260811 09:12:07.146905 26777 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:12:07 dist-test-slave-xt4k krb5kdc[25585](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439527, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:12:07.150452 26777 init.cc:377] Logged in from keytab as kudu/127.28.154.129@KRBTEST.COM (short username kudu)
I20260811 09:12:07.152150 26777 webserver.cc:460] Webserver started at http://127.28.154.129:36297/ using document root <none> and password file <none>
I20260811 09:12:07.153007 26777 fs_manager.cc:359] Metadata directory not provided
I20260811 09:12:07.153157 26777 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:12:07.159349 26777 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 09:12:07.162569 26847 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:12:07.164150 26777 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 09:12:07.164379 26777 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-0/wal
uuid: "153e73b493bf47509fd161e335705439"
format_stamp: "Formatted at 2026-08-11 09:11:43 on dist-test-slave-xt4k"
server_key: "a08bdbd47d5f29477d745c6e9f160c9d"
server_key_iv: "ae5122a0f882baf48ec9c9e6bd0c8b01"
server_key_version: "encryptionkey@0"
I20260811 09:12:07.165431 26777 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:12:07.191514 26777 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:12:07.196249 26777 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:12:07.196591 26777 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:12:07.197652 26777 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:12:07.199061 26777 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:12:07.199159 26777 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:12:07.199266 26777 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:12:07.199411 26777 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:12:07.234745 26777 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.129:41979
I20260811 09:12:07.234889 27038 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.129:41979 every 8 connection(s)
I20260811 09:12:07.236267 26777 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-0/data/info.pb
I20260811 09:12:07.245783 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 26777
I20260811 09:12:07.246587 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.154.130:37427
--local_ip_for_outbound_sockets=127.28.154.130
--tserver_master_addrs=127.28.154.190:40403
--webserver_port=45323
--webserver_interface=127.28.154.130
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.130.keytab
--principal=kudu/127.28.154.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:38289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 09:12:07 dist-test-slave-xt4k krb5kdc[25585](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439527, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.129@KRBTEST.COM for kudu/127.28.154.190@KRBTEST.COM
I20260811 09:12:07.262631 27044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:07.239410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:40403 (local address 127.28.154.129:45797)
0811 09:12:07.240225 (+ 815us) negotiation.cc:107] Waiting for socket to connect
0811 09:12:07.240284 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 09:12:07.242388 (+ 2104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:12:07.243958 (+ 1570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:12:07.243966 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:12:07.244617 (+ 651us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:12:07.245196 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:12:07.245215 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:12:07.247969 (+ 2754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:12:07.247974 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:12:07.249492 (+ 1518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:12:07.249508 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:12:07.250871 (+ 1363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:12:07.250877 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:12:07.251107 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:12:07.251814 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:12:07.251840 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:12:07.254381 (+ 2541us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:12:07.257961 (+ 3580us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:12:07.257971 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:12:07.257996 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:12:07.258599 (+ 603us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:12:07.259216 (+ 617us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:12:07.259219 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:12:07.259222 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:12:07.259430 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:12:07.260443 (+ 1013us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:12:07.260449 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:12:07.260795 (+ 346us) client_negotiation.cc:769] Sending connection context
0811 09:12:07.261065 (+ 270us) client_negotiation.cc:240] Negotiation successful
0811 09:12:07.261547 (+ 482us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":520,"thread_start_us":324,"threads_started":1}
I20260811 09:12:07.262907 27042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:07.239292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.129:45797 (local address 127.28.154.190:40403)
0811 09:12:07.239981 (+ 689us) server_negotiation.cc:206] Beginning negotiation
0811 09:12:07.239993 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 09:12:07.240529 (+ 536us) server_negotiation.cc:407] Connection header received
0811 09:12:07.242982 (+ 2453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:12:07.243001 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:12:07.243422 (+ 421us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:12:07.243775 (+ 353us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:12:07.245502 (+ 1727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:07.247728 (+ 2226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:07.249663 (+ 1935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:07.250610 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:07.250804 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:12:07.254758 (+ 3954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:12:07.254767 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:12:07.254785 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:12:07.254826 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:12:07.257790 (+ 2964us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:12:07.258797 (+ 1007us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:12:07.258808 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:12:07.258816 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:12:07.259094 (+ 278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:12:07.259551 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:12:07.259557 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:12:07.259560 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:12:07.260282 (+ 722us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:12:07.260538 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0811 09:12:07.261324 (+ 786us) server_negotiation.cc:299] Negotiation successful
0811 09:12:07.261758 (+ 434us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":491,"thread_start_us":327,"threads_started":1}
I20260811 09:12:07.266077 27039 heartbeater.cc:347] Connected to a master server at 127.28.154.190:40403
I20260811 09:12:07.266404 27039 heartbeater.cc:464] Registering TS with master...
I20260811 09:12:07.267320 27039 heartbeater.cc:511] Master 127.28.154.190:40403 requested a full tablet report, sending...
I20260811 09:12:07.270010 26712 ts_manager.cc:194] Registered new tserver with Master: 153e73b493bf47509fd161e335705439 (127.28.154.129:41979)
I20260811 09:12:07.271808 26712 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.154.129@KRBTEST.COM'} at 127.28.154.129:45797
W20260811 09:12:07.592721 27045 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:12:07.593006 27045 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:12:07.593132 27045 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:12:07.600451 27045 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:12:07.600571 27045 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:12:07.600720 27045 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.154.130
I20260811 09:12:07.610646 27045 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:38289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.130.keytab
--principal=kudu/127.28.154.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.154.130:37427
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.154.130
--webserver_port=45323
--tserver_master_addrs=127.28.154.190:40403
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.154.130
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:12:07.611989 27045 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:12:07.613260 27045 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:12:07.626451 27060 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:12:07.626499 27056 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:12:07.626933 27057 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:12:07.627240 27045 server_base.cc:1034] running on GCE node
I20260811 09:12:07.627846 27045 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:12:07.628854 27045 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:12:07.630055 27045 hybrid_clock.cc:648] HybridClock initialized: now 1786439527630005 us; error 58 us; skew 500 ppm
I20260811 09:12:07.630355 27045 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:12:07 dist-test-slave-xt4k krb5kdc[25585](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439527, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:12:07.633440 27045 init.cc:377] Logged in from keytab as kudu/127.28.154.130@KRBTEST.COM (short username kudu)
I20260811 09:12:07.634785 27045 webserver.cc:460] Webserver started at http://127.28.154.130:45323/ using document root <none> and password file <none>
I20260811 09:12:07.635443 27045 fs_manager.cc:359] Metadata directory not provided
I20260811 09:12:07.635558 27045 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:12:07.640928 27045 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260811 09:12:07.643740 27073 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:12:07.645211 27045 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 09:12:07.645380 27045 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-1/wal
uuid: "84334312b3254576991f3f49ecc134b1"
format_stamp: "Formatted at 2026-08-11 09:11:44 on dist-test-slave-xt4k"
server_key: "57769100a6c65364fdc9ec81b45f0a3a"
server_key_iv: "eff99c23b261da5a2ed4654f64797ee0"
server_key_version: "encryptionkey@0"
I20260811 09:12:07.646365 27045 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:12:07.661178 27045 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:12:07.665697 27045 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:12:07.665977 27045 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:12:07.666950 27045 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:12:07.668316 27045 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:12:07.668505 27045 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260811 09:12:07.668618 27045 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:12:07.668744 27045 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:12:07.700361 27045 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.130:37427
I20260811 09:12:07.700472 27275 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.130:37427 every 8 connection(s)
I20260811 09:12:07.701841 27045 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-1/data/info.pb
I20260811 09:12:07.711383 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 27045
I20260811 09:12:07.712109 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.154.131:38565
--local_ip_for_outbound_sockets=127.28.154.131
--tserver_master_addrs=127.28.154.190:40403
--webserver_port=43089
--webserver_interface=127.28.154.131
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.131.keytab
--principal=kudu/127.28.154.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:38289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 09:12:07 dist-test-slave-xt4k krb5kdc[25585](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439527, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.130@KRBTEST.COM for kudu/127.28.154.190@KRBTEST.COM
I20260811 09:12:07.726159 27042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:07.704380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.130:50573 (local address 127.28.154.190:40403)
0811 09:12:07.704570 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 09:12:07.704575 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:12:07.705854 (+ 1279us) server_negotiation.cc:407] Connection header received
0811 09:12:07.708511 (+ 2657us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:12:07.708516 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:12:07.708626 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:12:07.708725 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:12:07.710716 (+ 1991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:07.711749 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:07.714141 (+ 2392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:07.714748 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:07.714870 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:12:07.719264 (+ 4394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:12:07.719272 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:12:07.719277 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:12:07.719321 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:12:07.722744 (+ 3423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:12:07.723699 (+ 955us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:12:07.723703 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:12:07.723706 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:12:07.723843 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:12:07.724290 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:12:07.724294 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:12:07.724296 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:12:07.724651 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:12:07.724799 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0811 09:12:07.725532 (+ 733us) server_negotiation.cc:299] Negotiation successful
0811 09:12:07.725783 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 09:12:07.727203 27279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:07.704847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:40403 (local address 127.28.154.130:50573)
0811 09:12:07.705663 (+ 816us) negotiation.cc:107] Waiting for socket to connect
0811 09:12:07.705710 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 09:12:07.708162 (+ 2452us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:12:07.708996 (+ 834us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:12:07.709007 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:12:07.709725 (+ 718us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:12:07.710512 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:12:07.710537 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:12:07.711932 (+ 1395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:12:07.711939 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:12:07.713973 (+ 2034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:12:07.713989 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:12:07.714987 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:12:07.714994 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:12:07.715281 (+ 287us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:12:07.716060 (+ 779us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:12:07.716090 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:12:07.718988 (+ 2898us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:12:07.722922 (+ 3934us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:12:07.722932 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:12:07.722959 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:12:07.723554 (+ 595us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:12:07.723965 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:12:07.723968 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:12:07.723971 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:12:07.724165 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:12:07.724782 (+ 617us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:12:07.724790 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:12:07.725154 (+ 364us) client_negotiation.cc:769] Sending connection context
0811 09:12:07.725398 (+ 244us) client_negotiation.cc:240] Negotiation successful
0811 09:12:07.725978 (+ 580us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":507,"thread_start_us":296,"threads_started":1}
I20260811 09:12:07.728744 27276 heartbeater.cc:347] Connected to a master server at 127.28.154.190:40403
I20260811 09:12:07.729094 27276 heartbeater.cc:464] Registering TS with master...
I20260811 09:12:07.729990 27276 heartbeater.cc:511] Master 127.28.154.190:40403 requested a full tablet report, sending...
I20260811 09:12:07.731788 26712 ts_manager.cc:194] Registered new tserver with Master: 84334312b3254576991f3f49ecc134b1 (127.28.154.130:37427)
I20260811 09:12:07.732700 26712 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.154.130@KRBTEST.COM'} at 127.28.154.130:50573
W20260811 09:12:08.068775 27282 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:12:08.069073 27282 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:12:08.069216 27282 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:12:08.076671 27282 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:12:08.076800 27282 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:12:08.076946 27282 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.154.131
I20260811 09:12:08.087359 27282 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:38289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.131.keytab
--principal=kudu/127.28.154.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.154.131:38565
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.154.131
--webserver_port=43089
--tserver_master_addrs=127.28.154.190:40403
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.154.131
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:12:08.088761 27282 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:12:08.090073 27282 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:12:08.103147 27288 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:12:08.103081 27292 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:12:08.103118 27289 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:12:08.103878 27282 server_base.cc:1034] running on GCE node
I20260811 09:12:08.104539 27282 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:12:08.105576 27282 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:12:08.106827 27282 hybrid_clock.cc:648] HybridClock initialized: now 1786439528106756 us; error 55 us; skew 500 ppm
I20260811 09:12:08.107177 27282 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:12:08 dist-test-slave-xt4k krb5kdc[25585](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439528, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:12:08.110595 27282 init.cc:377] Logged in from keytab as kudu/127.28.154.131@KRBTEST.COM (short username kudu)
I20260811 09:12:08.112200 27282 webserver.cc:460] Webserver started at http://127.28.154.131:43089/ using document root <none> and password file <none>
I20260811 09:12:08.112962 27282 fs_manager.cc:359] Metadata directory not provided
I20260811 09:12:08.113103 27282 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:12:08.118654 27282 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 09:12:08.121573 27304 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:12:08.122936 27282 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 09:12:08.123106 27282 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-2/wal
uuid: "82c897eb04564e7d8870a909b3082d02"
format_stamp: "Formatted at 2026-08-11 09:11:45 on dist-test-slave-xt4k"
server_key: "a90f2eeafb5bab0abe030a448def00db"
server_key_iv: "6895a616d1b1d17f28b8186124151e9f"
server_key_version: "encryptionkey@0"
I20260811 09:12:08.124092 27282 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:12:08.144253 27282 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:12:08.148492 27282 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:12:08.148801 27282 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:12:08.149922 27282 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:12:08.151403 27282 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:12:08.151503 27282 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:12:08.151623 27282 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:12:08.151692 27282 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:12:08.186982 27282 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.131:38565
I20260811 09:12:08.187047 27488 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.131:38565 every 8 connection(s)
I20260811 09:12:08.188484 27282 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-2/data/info.pb
I20260811 09:12:08.196797 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 27282
Aug 11 09:12:08 dist-test-slave-xt4k krb5kdc[25585](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439528, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.131@KRBTEST.COM for kudu/127.28.154.190@KRBTEST.COM
I20260811 09:12:08.210672 27042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:08.191358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.131:45161 (local address 127.28.154.190:40403)
0811 09:12:08.191551 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 09:12:08.191555 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:12:08.192813 (+ 1258us) server_negotiation.cc:407] Connection header received
0811 09:12:08.194725 (+ 1912us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:12:08.194729 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:12:08.194804 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:12:08.194922 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:12:08.196683 (+ 1761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:08.197520 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:08.199654 (+ 2134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:08.200371 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:08.200555 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:12:08.204635 (+ 4080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:12:08.204640 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:12:08.204643 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:12:08.204678 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:12:08.207371 (+ 2693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:12:08.208200 (+ 829us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:12:08.208204 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:12:08.208206 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:12:08.208379 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:12:08.208831 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:12:08.208838 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:12:08.208840 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:12:08.209147 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:12:08.209295 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0811 09:12:08.210061 (+ 766us) server_negotiation.cc:299] Negotiation successful
0811 09:12:08.210302 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 09:12:08.211763 27493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:08.191757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:40403 (local address 127.28.154.131:45161)
0811 09:12:08.192593 (+ 836us) negotiation.cc:107] Waiting for socket to connect
0811 09:12:08.192659 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0811 09:12:08.194420 (+ 1761us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:12:08.195145 (+ 725us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:12:08.195154 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:12:08.195717 (+ 563us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:12:08.196438 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:12:08.196468 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:12:08.197739 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:12:08.197745 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:12:08.199488 (+ 1743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:12:08.199501 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:12:08.200632 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:12:08.200638 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:12:08.200930 (+ 292us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:12:08.201736 (+ 806us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:12:08.201768 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:12:08.204403 (+ 2635us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:12:08.207545 (+ 3142us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:12:08.207557 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:12:08.207575 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:12:08.208062 (+ 487us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:12:08.208492 (+ 430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:12:08.208498 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:12:08.208502 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:12:08.208711 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:12:08.209274 (+ 563us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:12:08.209282 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:12:08.209669 (+ 387us) client_negotiation.cc:769] Sending connection context
0811 09:12:08.209933 (+ 264us) client_negotiation.cc:240] Negotiation successful
0811 09:12:08.210509 (+ 576us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":302,"threads_started":1}
I20260811 09:12:08.213449 27491 heartbeater.cc:347] Connected to a master server at 127.28.154.190:40403
I20260811 09:12:08.213830 27491 heartbeater.cc:464] Registering TS with master...
I20260811 09:12:08.214341 27496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:08.197810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56468 (local address 127.28.154.190:40403)
0811 09:12:08.198584 (+ 774us) server_negotiation.cc:206] Beginning negotiation
0811 09:12:08.198589 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:12:08.198608 (+ 19us) server_negotiation.cc:407] Connection header received
0811 09:12:08.198660 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:12:08.198665 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:12:08.198746 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:12:08.198858 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:12:08.200885 (+ 2027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:08.201915 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:08.203625 (+ 1710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:08.204222 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:08.204376 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:12:08.208009 (+ 3633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:12:08.208014 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:12:08.208018 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:12:08.208052 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:12:08.210669 (+ 2617us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:12:08.211744 (+ 1075us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:12:08.211755 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:12:08.211758 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:12:08.211938 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:12:08.212706 (+ 768us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:12:08.212710 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:12:08.212712 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:12:08.213068 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:12:08.213288 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0811 09:12:08.213700 (+ 412us) server_negotiation.cc:299] Negotiation successful
0811 09:12:08.213941 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":645,"thread_start_us":520,"threads_started":1}
I20260811 09:12:08.214797 27491 heartbeater.cc:511] Master 127.28.154.190:40403 requested a full tablet report, sending...
I20260811 09:12:08.216801 26711 ts_manager.cc:194] Registered new tserver with Master: 82c897eb04564e7d8870a909b3082d02 (127.28.154.131:38565)
I20260811 09:12:08.217921 26711 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.154.131@KRBTEST.COM'} at 127.28.154.131:45161
I20260811 09:12:08.230110 29290 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 09:12:08.238366 27496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:08.231082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56472 (local address 127.28.154.190:40403)
0811 09:12:08.231292 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0811 09:12:08.231297 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:12:08.231784 (+ 487us) server_negotiation.cc:407] Connection header received
0811 09:12:08.232018 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:12:08.232024 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:12:08.232191 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:12:08.232383 (+ 192us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:12:08.232874 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:08.233642 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:08.235722 (+ 2080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:08.236438 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:08.236580 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:12:08.237229 (+ 649us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:12:08.237611 (+ 382us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0811 09:12:08.238041 (+ 430us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:56472: token expired
Metrics: {"server-negotiator.queue_time_us":62}
W20260811 09:12:08.238924 27496 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:56472: token expired
W20260811 09:12:08.239169 26564 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.28.154.190:40403)
I20260811 09:12:08.256647 27496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:08.239774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56480 (local address 127.28.154.190:40403)
0811 09:12:08.239957 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0811 09:12:08.239963 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:12:08.239982 (+ 19us) server_negotiation.cc:407] Connection header received
0811 09:12:08.240084 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:12:08.240089 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:12:08.240162 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:12:08.240267 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:12:08.241940 (+ 1673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:08.242764 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:08.244908 (+ 2144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:08.245355 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:08.245513 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:12:08.249873 (+ 4360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:12:08.249879 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:12:08.249882 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:12:08.249926 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:12:08.253353 (+ 3427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:12:08.254433 (+ 1080us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:12:08.254439 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:12:08.254441 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:12:08.254679 (+ 238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:12:08.255121 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:12:08.255125 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:12:08.255128 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:12:08.255510 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:12:08.255730 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0811 09:12:08.256132 (+ 402us) server_negotiation.cc:299] Negotiation successful
0811 09:12:08.256327 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 09:12:08.260516 26711 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56480:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 09:12:08.267946 26711 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 09:12:08.276278 27039 heartbeater.cc:503] Master 127.28.154.190:40403 was elected leader, sending a full tablet report...
I20260811 09:12:08.291853 27504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:08.281896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.131:38565 (local address 127.0.0.1:51260)
0811 09:12:08.282764 (+ 868us) negotiation.cc:107] Waiting for socket to connect
0811 09:12:08.282806 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 09:12:08.283001 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:12:08.284051 (+ 1050us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:12:08.284058 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:12:08.284085 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:12:08.284486 (+ 401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:12:08.284498 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:12:08.286860 (+ 2362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:12:08.286865 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:12:08.289294 (+ 2429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:12:08.289311 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:12:08.291096 (+ 1785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:12:08.291102 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:12:08.291277 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:12:08.291317 (+ 40us) client_negotiation.cc:769] Sending connection context
0811 09:12:08.291418 (+ 101us) client_negotiation.cc:240] Negotiation successful
0811 09:12:08.291563 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":522,"thread_start_us":1068,"threads_started":1}
I20260811 09:12:08.292925 27506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:08.282387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51260 (local address 127.28.154.131:38565)
0811 09:12:08.283224 (+ 837us) server_negotiation.cc:206] Beginning negotiation
0811 09:12:08.283232 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:12:08.283268 (+ 36us) server_negotiation.cc:407] Connection header received
0811 09:12:08.283367 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:12:08.283374 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:12:08.283666 (+ 292us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:12:08.284041 (+ 375us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:12:08.284754 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:08.286661 (+ 1907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:08.289516 (+ 2855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:08.290851 (+ 1335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:08.292165 (+ 1314us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:12:08.292388 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0811 09:12:08.292476 (+ 88us) server_negotiation.cc:299] Negotiation successful
0811 09:12:08.292617 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":670,"thread_start_us":922,"threads_started":1}
I20260811 09:12:08.295140 27507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:08.282517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.129:41979 (local address 127.0.0.1:44002)
0811 09:12:08.286852 (+ 4335us) negotiation.cc:107] Waiting for socket to connect
0811 09:12:08.286877 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 09:12:08.287011 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:12:08.288022 (+ 1011us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:12:08.288027 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:12:08.288056 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:12:08.288573 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:12:08.288586 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:12:08.290453 (+ 1867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:12:08.290458 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:12:08.292629 (+ 2171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:12:08.292645 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:12:08.294351 (+ 1706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:12:08.294370 (+ 19us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:12:08.294536 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:12:08.294574 (+ 38us) client_negotiation.cc:769] Sending connection context
0811 09:12:08.294647 (+ 73us) client_negotiation.cc:240] Negotiation successful
0811 09:12:08.294790 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4207,"thread_start_us":4063,"threads_started":1}
I20260811 09:12:08.296139 27509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:08.283244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44002 (local address 127.28.154.129:41979)
0811 09:12:08.283802 (+ 558us) server_negotiation.cc:206] Beginning negotiation
0811 09:12:08.283811 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 09:12:08.287152 (+ 3341us) server_negotiation.cc:407] Connection header received
0811 09:12:08.287262 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:12:08.287269 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:12:08.287523 (+ 254us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:12:08.287751 (+ 228us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:12:08.288784 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:08.290234 (+ 1450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:08.292806 (+ 2572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:08.293978 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:08.295401 (+ 1423us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:12:08.295565 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0811 09:12:08.295639 (+ 74us) server_negotiation.cc:299] Negotiation successful
0811 09:12:08.295814 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":394,"thread_start_us":606,"threads_started":1}
I20260811 09:12:08.296502 27511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:08.287940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.130:37427 (local address 127.0.0.1:54512)
0811 09:12:08.288586 (+ 646us) negotiation.cc:107] Waiting for socket to connect
0811 09:12:08.288603 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 09:12:08.288728 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:12:08.289302 (+ 574us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:12:08.289307 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:12:08.289332 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:12:08.289747 (+ 415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:12:08.289770 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:12:08.291697 (+ 1927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:12:08.291702 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:12:08.293653 (+ 1951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:12:08.293667 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:12:08.295734 (+ 2067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:12:08.295741 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:12:08.295910 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:12:08.295944 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 09:12:08.296041 (+ 97us) client_negotiation.cc:240] Negotiation successful
0811 09:12:08.296174 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":310,"threads_started":1}
I20260811 09:12:08.297827 27510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:08.287296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54512 (local address 127.28.154.130:37427)
0811 09:12:08.287941 (+ 645us) server_negotiation.cc:206] Beginning negotiation
0811 09:12:08.287960 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 09:12:08.288720 (+ 760us) server_negotiation.cc:407] Connection header received
0811 09:12:08.288860 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:12:08.288869 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:12:08.289099 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:12:08.289295 (+ 196us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:12:08.290000 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:08.291497 (+ 1497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:08.293829 (+ 2332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:08.295224 (+ 1395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:08.295382 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:12:08.295565 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0811 09:12:08.296112 (+ 547us) server_negotiation.cc:299] Negotiation successful
0811 09:12:08.297524 (+ 1412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":480,"thread_start_us":330,"threads_started":1}
I20260811 09:12:08.297350 27381 tablet_service.cc:1467] Processing CreateTablet for tablet c4a7d334176e4e748d0700ec8b543b73 (DEFAULT_TABLE table=test-table [id=2c96dc792abb4509b7308084bdb304e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:12:08.298897 27381 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c4a7d334176e4e748d0700ec8b543b73. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:12:08.300012 26926 tablet_service.cc:1467] Processing CreateTablet for tablet c4a7d334176e4e748d0700ec8b543b73 (DEFAULT_TABLE table=test-table [id=2c96dc792abb4509b7308084bdb304e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:12:08.301381 26926 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c4a7d334176e4e748d0700ec8b543b73. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:12:08.301466 27169 tablet_service.cc:1467] Processing CreateTablet for tablet c4a7d334176e4e748d0700ec8b543b73 (DEFAULT_TABLE table=test-table [id=2c96dc792abb4509b7308084bdb304e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:12:08.302811 27169 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c4a7d334176e4e748d0700ec8b543b73. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:12:08.314657 27512 tablet_bootstrap.cc:492] T c4a7d334176e4e748d0700ec8b543b73 P 153e73b493bf47509fd161e335705439: Bootstrap starting.
I20260811 09:12:08.315280 27515 tablet_bootstrap.cc:492] T c4a7d334176e4e748d0700ec8b543b73 P 82c897eb04564e7d8870a909b3082d02: Bootstrap starting.
I20260811 09:12:08.318565 27516 tablet_bootstrap.cc:492] T c4a7d334176e4e748d0700ec8b543b73 P 84334312b3254576991f3f49ecc134b1: Bootstrap starting.
I20260811 09:12:08.318657 27512 tablet_bootstrap.cc:654] T c4a7d334176e4e748d0700ec8b543b73 P 153e73b493bf47509fd161e335705439: Neither blocks nor log segments found. Creating new log.
I20260811 09:12:08.318658 27515 tablet_bootstrap.cc:654] T c4a7d334176e4e748d0700ec8b543b73 P 82c897eb04564e7d8870a909b3082d02: Neither blocks nor log segments found. Creating new log.
I20260811 09:12:08.319969 27512 log.cc:824] T c4a7d334176e4e748d0700ec8b543b73 P 153e73b493bf47509fd161e335705439: Log is configured to *not* fsync() on all Append() calls
I20260811 09:12:08.319962 27515 log.cc:824] T c4a7d334176e4e748d0700ec8b543b73 P 82c897eb04564e7d8870a909b3082d02: Log is configured to *not* fsync() on all Append() calls
I20260811 09:12:08.322116 27516 tablet_bootstrap.cc:654] T c4a7d334176e4e748d0700ec8b543b73 P 84334312b3254576991f3f49ecc134b1: Neither blocks nor log segments found. Creating new log.
I20260811 09:12:08.323457 27516 log.cc:824] T c4a7d334176e4e748d0700ec8b543b73 P 84334312b3254576991f3f49ecc134b1: Log is configured to *not* fsync() on all Append() calls
I20260811 09:12:08.323695 27512 tablet_bootstrap.cc:492] T c4a7d334176e4e748d0700ec8b543b73 P 153e73b493bf47509fd161e335705439: No bootstrap required, opened a new log
I20260811 09:12:08.323982 27515 tablet_bootstrap.cc:492] T c4a7d334176e4e748d0700ec8b543b73 P 82c897eb04564e7d8870a909b3082d02: No bootstrap required, opened a new log
I20260811 09:12:08.324000 27512 ts_tablet_manager.cc:1397] T c4a7d334176e4e748d0700ec8b543b73 P 153e73b493bf47509fd161e335705439: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260811 09:12:08.324378 27515 ts_tablet_manager.cc:1397] T c4a7d334176e4e748d0700ec8b543b73 P 82c897eb04564e7d8870a909b3082d02: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260811 09:12:08.325989 27516 tablet_bootstrap.cc:492] T c4a7d334176e4e748d0700ec8b543b73 P 84334312b3254576991f3f49ecc134b1: No bootstrap required, opened a new log
I20260811 09:12:08.326233 27516 ts_tablet_manager.cc:1397] T c4a7d334176e4e748d0700ec8b543b73 P 84334312b3254576991f3f49ecc134b1: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260811 09:12:08.333882 27516 raft_consensus.cc:348] T c4a7d334176e4e748d0700ec8b543b73 P 84334312b3254576991f3f49ecc134b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82c897eb04564e7d8870a909b3082d02" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 38565 } } peers { permanent_uuid: "153e73b493bf47509fd161e335705439" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41979 } } peers { permanent_uuid: "84334312b3254576991f3f49ecc134b1" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 37427 } }
I20260811 09:12:08.334365 27516 raft_consensus.cc:374] T c4a7d334176e4e748d0700ec8b543b73 P 84334312b3254576991f3f49ecc134b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:12:08.334496 27516 raft_consensus.cc:729] T c4a7d334176e4e748d0700ec8b543b73 P 84334312b3254576991f3f49ecc134b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84334312b3254576991f3f49ecc134b1, State: Initialized, Role: FOLLOWER
I20260811 09:12:08.334658 27515 raft_consensus.cc:348] T c4a7d334176e4e748d0700ec8b543b73 P 82c897eb04564e7d8870a909b3082d02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82c897eb04564e7d8870a909b3082d02" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 38565 } } peers { permanent_uuid: "153e73b493bf47509fd161e335705439" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41979 } } peers { permanent_uuid: "84334312b3254576991f3f49ecc134b1" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 37427 } }
I20260811 09:12:08.334658 27512 raft_consensus.cc:348] T c4a7d334176e4e748d0700ec8b543b73 P 153e73b493bf47509fd161e335705439 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82c897eb04564e7d8870a909b3082d02" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 38565 } } peers { permanent_uuid: "153e73b493bf47509fd161e335705439" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41979 } } peers { permanent_uuid: "84334312b3254576991f3f49ecc134b1" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 37427 } }
I20260811 09:12:08.335083 27512 raft_consensus.cc:374] T c4a7d334176e4e748d0700ec8b543b73 P 153e73b493bf47509fd161e335705439 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:12:08.335083 27515 raft_consensus.cc:374] T c4a7d334176e4e748d0700ec8b543b73 P 82c897eb04564e7d8870a909b3082d02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:12:08.335186 27515 raft_consensus.cc:729] T c4a7d334176e4e748d0700ec8b543b73 P 82c897eb04564e7d8870a909b3082d02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82c897eb04564e7d8870a909b3082d02, State: Initialized, Role: FOLLOWER
I20260811 09:12:08.335191 27512 raft_consensus.cc:729] T c4a7d334176e4e748d0700ec8b543b73 P 153e73b493bf47509fd161e335705439 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 153e73b493bf47509fd161e335705439, State: Initialized, Role: FOLLOWER
I20260811 09:12:08.335130 27516 consensus_queue.cc:260] T c4a7d334176e4e748d0700ec8b543b73 P 84334312b3254576991f3f49ecc134b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82c897eb04564e7d8870a909b3082d02" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 38565 } } peers { permanent_uuid: "153e73b493bf47509fd161e335705439" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41979 } } peers { permanent_uuid: "84334312b3254576991f3f49ecc134b1" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 37427 } }
I20260811 09:12:08.335813 27515 consensus_queue.cc:260] T c4a7d334176e4e748d0700ec8b543b73 P 82c897eb04564e7d8870a909b3082d02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82c897eb04564e7d8870a909b3082d02" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 38565 } } peers { permanent_uuid: "153e73b493bf47509fd161e335705439" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41979 } } peers { permanent_uuid: "84334312b3254576991f3f49ecc134b1" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 37427 } }
I20260811 09:12:08.335778 27512 consensus_queue.cc:260] T c4a7d334176e4e748d0700ec8b543b73 P 153e73b493bf47509fd161e335705439 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82c897eb04564e7d8870a909b3082d02" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 38565 } } peers { permanent_uuid: "153e73b493bf47509fd161e335705439" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41979 } } peers { permanent_uuid: "84334312b3254576991f3f49ecc134b1" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 37427 } }
I20260811 09:12:08.336485 27276 heartbeater.cc:503] Master 127.28.154.190:40403 was elected leader, sending a full tablet report...
I20260811 09:12:08.337733 27515 ts_tablet_manager.cc:1428] T c4a7d334176e4e748d0700ec8b543b73 P 82c897eb04564e7d8870a909b3082d02: Time spent starting tablet: real 0.013s user 0.012s sys 0.001s
I20260811 09:12:08.337776 27491 heartbeater.cc:503] Master 127.28.154.190:40403 was elected leader, sending a full tablet report...
I20260811 09:12:08.339258 27512 ts_tablet_manager.cc:1428] T c4a7d334176e4e748d0700ec8b543b73 P 153e73b493bf47509fd161e335705439: Time spent starting tablet: real 0.015s user 0.012s sys 0.004s
I20260811 09:12:08.340458 27516 ts_tablet_manager.cc:1428] T c4a7d334176e4e748d0700ec8b543b73 P 84334312b3254576991f3f49ecc134b1: Time spent starting tablet: real 0.014s user 0.012s sys 0.000s
I20260811 09:12:08.424583 27525 raft_consensus.cc:482] T c4a7d334176e4e748d0700ec8b543b73 P 153e73b493bf47509fd161e335705439 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:12:08.424932 27525 raft_consensus.cc:504] T c4a7d334176e4e748d0700ec8b543b73 P 153e73b493bf47509fd161e335705439 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82c897eb04564e7d8870a909b3082d02" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 38565 } } peers { permanent_uuid: "153e73b493bf47509fd161e335705439" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41979 } } peers { permanent_uuid: "84334312b3254576991f3f49ecc134b1" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 37427 } }
I20260811 09:12:08.426748 27525 leader_election.cc:288] T c4a7d334176e4e748d0700ec8b543b73 P 153e73b493bf47509fd161e335705439 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 82c897eb04564e7d8870a909b3082d02 (127.28.154.131:38565), 84334312b3254576991f3f49ecc134b1 (127.28.154.130:37427)
I20260811 09:12:08.434443 27506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:08.427572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.129:49775 (local address 127.28.154.131:38565)
0811 09:12:08.427795 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0811 09:12:08.427801 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:12:08.428565 (+ 764us) server_negotiation.cc:407] Connection header received
0811 09:12:08.428920 (+ 355us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:12:08.428925 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:12:08.428998 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:12:08.429078 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:12:08.429753 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:08.430477 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:08.432426 (+ 1949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:08.433378 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:08.433568 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:12:08.433660 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 09:12:08.434062 (+ 402us) server_negotiation.cc:299] Negotiation successful
0811 09:12:08.434191 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260811 09:12:08.434443 27510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:08.427501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.129:46387 (local address 127.28.154.130:37427)
0811 09:12:08.427768 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0811 09:12:08.427775 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:12:08.428728 (+ 953us) server_negotiation.cc:407] Connection header received
0811 09:12:08.428900 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:12:08.428904 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:12:08.428987 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:12:08.429078 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:12:08.429710 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:08.430440 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:08.432454 (+ 2014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:08.433393 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:08.433610 (+ 217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:12:08.433696 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 09:12:08.434051 (+ 355us) server_negotiation.cc:299] Negotiation successful
0811 09:12:08.434189 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260811 09:12:08.434666 27531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:08.427569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.130:37427 (local address 127.28.154.129:46387)
0811 09:12:08.428603 (+ 1034us) negotiation.cc:107] Waiting for socket to connect
0811 09:12:08.428622 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 09:12:08.428771 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:12:08.429124 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:12:08.429128 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:12:08.429153 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:12:08.429534 (+ 381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:12:08.429549 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:12:08.430575 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:12:08.430580 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:12:08.432272 (+ 1692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:12:08.432285 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:12:08.433698 (+ 1413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:12:08.433705 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:12:08.433886 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:12:08.433922 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 09:12:08.434003 (+ 81us) client_negotiation.cc:240] Negotiation successful
0811 09:12:08.434160 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":908,"mutex_wait_us":124,"thread_start_us":460,"threads_started":1}
I20260811 09:12:08.434676 27529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:08.427268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.131:38565 (local address 127.28.154.129:49775)
0811 09:12:08.428429 (+ 1161us) negotiation.cc:107] Waiting for socket to connect
0811 09:12:08.428450 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 09:12:08.428637 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:12:08.429124 (+ 487us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:12:08.429128 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:12:08.429221 (+ 93us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:12:08.429611 (+ 390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:12:08.429624 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:12:08.430613 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:12:08.430617 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:12:08.432259 (+ 1642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:12:08.432273 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:12:08.433645 (+ 1372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:12:08.433652 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:12:08.433872 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:12:08.433918 (+ 46us) client_negotiation.cc:769] Sending connection context
0811 09:12:08.434010 (+ 92us) client_negotiation.cc:240] Negotiation successful
0811 09:12:08.434148 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":993,"mutex_wait_us":53,"spinlock_wait_cycles":24448,"thread_start_us":716,"threads_started":1}
I20260811 09:12:08.435889 27415 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4a7d334176e4e748d0700ec8b543b73" candidate_uuid: "153e73b493bf47509fd161e335705439" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "82c897eb04564e7d8870a909b3082d02" is_pre_election: true
I20260811 09:12:08.436004 27198 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4a7d334176e4e748d0700ec8b543b73" candidate_uuid: "153e73b493bf47509fd161e335705439" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84334312b3254576991f3f49ecc134b1" is_pre_election: true
I20260811 09:12:08.436434 27415 raft_consensus.cc:2454] T c4a7d334176e4e748d0700ec8b543b73 P 82c897eb04564e7d8870a909b3082d02 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 153e73b493bf47509fd161e335705439 in term 0.
I20260811 09:12:08.436532 27198 raft_consensus.cc:2454] T c4a7d334176e4e748d0700ec8b543b73 P 84334312b3254576991f3f49ecc134b1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 153e73b493bf47509fd161e335705439 in term 0.
I20260811 09:12:08.437178 26851 leader_election.cc:302] T c4a7d334176e4e748d0700ec8b543b73 P 153e73b493bf47509fd161e335705439 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 153e73b493bf47509fd161e335705439, 82c897eb04564e7d8870a909b3082d02; no voters:
I20260811 09:12:08.437641 27525 raft_consensus.cc:2781] T c4a7d334176e4e748d0700ec8b543b73 P 153e73b493bf47509fd161e335705439 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 09:12:08.437800 27525 raft_consensus.cc:482] T c4a7d334176e4e748d0700ec8b543b73 P 153e73b493bf47509fd161e335705439 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 09:12:08.437903 27525 raft_consensus.cc:3037] T c4a7d334176e4e748d0700ec8b543b73 P 153e73b493bf47509fd161e335705439 [term 0 FOLLOWER]: Advancing to term 1
W20260811 09:12:08.440111 27492 tablet.cc:2369] T c4a7d334176e4e748d0700ec8b543b73 P 82c897eb04564e7d8870a909b3082d02: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 09:12:08.441210 27525 raft_consensus.cc:504] T c4a7d334176e4e748d0700ec8b543b73 P 153e73b493bf47509fd161e335705439 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82c897eb04564e7d8870a909b3082d02" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 38565 } } peers { permanent_uuid: "153e73b493bf47509fd161e335705439" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41979 } } peers { permanent_uuid: "84334312b3254576991f3f49ecc134b1" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 37427 } }
I20260811 09:12:08.441821 27525 leader_election.cc:288] T c4a7d334176e4e748d0700ec8b543b73 P 153e73b493bf47509fd161e335705439 [CANDIDATE]: Term 1 election: Requested vote from peers 82c897eb04564e7d8870a909b3082d02 (127.28.154.131:38565), 84334312b3254576991f3f49ecc134b1 (127.28.154.130:37427)
I20260811 09:12:08.442422 27198 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4a7d334176e4e748d0700ec8b543b73" candidate_uuid: "153e73b493bf47509fd161e335705439" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "84334312b3254576991f3f49ecc134b1"
I20260811 09:12:08.442672 27198 raft_consensus.cc:3037] T c4a7d334176e4e748d0700ec8b543b73 P 84334312b3254576991f3f49ecc134b1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:12:08.442656 27415 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4a7d334176e4e748d0700ec8b543b73" candidate_uuid: "153e73b493bf47509fd161e335705439" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "82c897eb04564e7d8870a909b3082d02"
I20260811 09:12:08.442967 27415 raft_consensus.cc:3037] T c4a7d334176e4e748d0700ec8b543b73 P 82c897eb04564e7d8870a909b3082d02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:12:08.446452 27198 raft_consensus.cc:2454] T c4a7d334176e4e748d0700ec8b543b73 P 84334312b3254576991f3f49ecc134b1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 153e73b493bf47509fd161e335705439 in term 1.
I20260811 09:12:08.446451 27415 raft_consensus.cc:2454] T c4a7d334176e4e748d0700ec8b543b73 P 82c897eb04564e7d8870a909b3082d02 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 153e73b493bf47509fd161e335705439 in term 1.
I20260811 09:12:08.447189 26851 leader_election.cc:302] T c4a7d334176e4e748d0700ec8b543b73 P 153e73b493bf47509fd161e335705439 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 153e73b493bf47509fd161e335705439, 82c897eb04564e7d8870a909b3082d02; no voters:
I20260811 09:12:08.447541 27525 raft_consensus.cc:2781] T c4a7d334176e4e748d0700ec8b543b73 P 153e73b493bf47509fd161e335705439 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:12:08.448136 27525 raft_consensus.cc:686] T c4a7d334176e4e748d0700ec8b543b73 P 153e73b493bf47509fd161e335705439 [term 1 LEADER]: Becoming Leader. State: Replica: 153e73b493bf47509fd161e335705439, State: Running, Role: LEADER
I20260811 09:12:08.448680 27525 consensus_queue.cc:237] T c4a7d334176e4e748d0700ec8b543b73 P 153e73b493bf47509fd161e335705439 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82c897eb04564e7d8870a909b3082d02" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 38565 } } peers { permanent_uuid: "153e73b493bf47509fd161e335705439" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41979 } } peers { permanent_uuid: "84334312b3254576991f3f49ecc134b1" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 37427 } }
W20260811 09:12:08.453714 27277 tablet.cc:2369] T c4a7d334176e4e748d0700ec8b543b73 P 84334312b3254576991f3f49ecc134b1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 09:12:08.453881 26710 catalog_manager.cc:5495] T c4a7d334176e4e748d0700ec8b543b73 P 153e73b493bf47509fd161e335705439 reported cstate change: term changed from 0 to 1, leader changed from <none> to 153e73b493bf47509fd161e335705439 (127.28.154.129). New cstate: current_term: 1 leader_uuid: "153e73b493bf47509fd161e335705439" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82c897eb04564e7d8870a909b3082d02" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 38565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "153e73b493bf47509fd161e335705439" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41979 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "84334312b3254576991f3f49ecc134b1" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 37427 } health_report { overall_health: UNKNOWN } } }
I20260811 09:12:08.487982 27509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:08.482499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44012 (local address 127.28.154.129:41979)
0811 09:12:08.482780 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0811 09:12:08.482787 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:12:08.482808 (+ 21us) server_negotiation.cc:407] Connection header received
0811 09:12:08.483071 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:12:08.483075 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:12:08.483156 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:12:08.483320 (+ 164us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:12:08.483747 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:08.484487 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:08.485803 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:08.486230 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:08.486340 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:12:08.486856 (+ 516us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:12:08.487377 (+ 521us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:12:08.487534 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0811 09:12:08.487635 (+ 101us) server_negotiation.cc:299] Negotiation successful
0811 09:12:08.487717 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
W20260811 09:12:08.488646 27041 tablet.cc:2369] T c4a7d334176e4e748d0700ec8b543b73 P 153e73b493bf47509fd161e335705439: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 09:12:08.496315 27415 raft_consensus.cc:1261] T c4a7d334176e4e748d0700ec8b543b73 P 82c897eb04564e7d8870a909b3082d02 [term 1 FOLLOWER]: Refusing update from remote peer 153e73b493bf47509fd161e335705439: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:12:08.496316 27198 raft_consensus.cc:1261] T c4a7d334176e4e748d0700ec8b543b73 P 84334312b3254576991f3f49ecc134b1 [term 1 FOLLOWER]: Refusing update from remote peer 153e73b493bf47509fd161e335705439: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:12:08.497400 27533 consensus_queue.cc:1035] T c4a7d334176e4e748d0700ec8b543b73 P 153e73b493bf47509fd161e335705439 [LEADER]: Connected to new peer: Peer: permanent_uuid: "84334312b3254576991f3f49ecc134b1" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 37427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:12:08.497792 27525 consensus_queue.cc:1035] T c4a7d334176e4e748d0700ec8b543b73 P 153e73b493bf47509fd161e335705439 [LEADER]: Connected to new peer: Peer: permanent_uuid: "82c897eb04564e7d8870a909b3082d02" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 38565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:12:08.510774 27538 mvcc.cc:205] Tried to move back new op lower bound from 7317256308711268352 to 7317256308530098176. Current Snapshot: MvccSnapshot[applied={T|T < 7317256308711268352}]
I20260811 09:12:08.524694 26712 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56480:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:12:08.524962 26712 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56480:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:12:08.528637 26712 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b740e22f6c334fb38cdc579e01606268: Sending DeleteTablet for 3 replicas of tablet c4a7d334176e4e748d0700ec8b543b73
I20260811 09:12:08.529866 27381 tablet_service.cc:1514] Processing DeleteTablet for tablet c4a7d334176e4e748d0700ec8b543b73 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:12:08 UTC) from {username='kudu', principal='kudu/127.28.154.190@KRBTEST.COM'} at 127.0.0.1:51260
I20260811 09:12:08.530223 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 26777
I20260811 09:12:08.530272 26926 tablet_service.cc:1514] Processing DeleteTablet for tablet c4a7d334176e4e748d0700ec8b543b73 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:12:08 UTC) from {username='kudu', principal='kudu/127.28.154.190@KRBTEST.COM'} at 127.0.0.1:44002
I20260811 09:12:08.530987 27169 tablet_service.cc:1514] Processing DeleteTablet for tablet c4a7d334176e4e748d0700ec8b543b73 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:12:08 UTC) from {username='kudu', principal='kudu/127.28.154.190@KRBTEST.COM'} at 127.0.0.1:54512
I20260811 09:12:08.539863 27561 tablet_replica.cc:330] T c4a7d334176e4e748d0700ec8b543b73 P 84334312b3254576991f3f49ecc134b1: stopping tablet replica
I20260811 09:12:08.541071 27561 raft_consensus.cc:2229] T c4a7d334176e4e748d0700ec8b543b73 P 84334312b3254576991f3f49ecc134b1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:12:08.542330 27561 raft_consensus.cc:2258] T c4a7d334176e4e748d0700ec8b543b73 P 84334312b3254576991f3f49ecc134b1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:12:08.545113 27554 tablet_replica.cc:330] T c4a7d334176e4e748d0700ec8b543b73 P 82c897eb04564e7d8870a909b3082d02: stopping tablet replica
I20260811 09:12:08.545567 27554 raft_consensus.cc:2229] T c4a7d334176e4e748d0700ec8b543b73 P 82c897eb04564e7d8870a909b3082d02 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:12:08.546090 27554 raft_consensus.cc:2258] T c4a7d334176e4e748d0700ec8b543b73 P 82c897eb04564e7d8870a909b3082d02 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:12:08.548224 27561 ts_tablet_manager.cc:1905] T c4a7d334176e4e748d0700ec8b543b73 P 84334312b3254576991f3f49ecc134b1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:12:08.549863 27554 ts_tablet_manager.cc:1905] T c4a7d334176e4e748d0700ec8b543b73 P 82c897eb04564e7d8870a909b3082d02: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:12:08.553390 27559 tablet_replica.cc:330] T c4a7d334176e4e748d0700ec8b543b73 P 153e73b493bf47509fd161e335705439: stopping tablet replica
I20260811 09:12:08.553984 27559 raft_consensus.cc:2229] T c4a7d334176e4e748d0700ec8b543b73 P 153e73b493bf47509fd161e335705439 [term 1 LEADER]: Raft consensus shutting down.
I20260811 09:12:08.554893 27559 raft_consensus.cc:2258] T c4a7d334176e4e748d0700ec8b543b73 P 153e73b493bf47509fd161e335705439 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:12:08.555776 27554 ts_tablet_manager.cc:1918] T c4a7d334176e4e748d0700ec8b543b73 P 82c897eb04564e7d8870a909b3082d02: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:12:08.556121 27554 log.cc:1196] T c4a7d334176e4e748d0700ec8b543b73 P 82c897eb04564e7d8870a909b3082d02: Deleting WAL directory at /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-2/wal/wals/c4a7d334176e4e748d0700ec8b543b73
I20260811 09:12:08.556800 27554 ts_tablet_manager.cc:1939] T c4a7d334176e4e748d0700ec8b543b73 P 82c897eb04564e7d8870a909b3082d02: Deleting consensus metadata
I20260811 09:12:08.557279 27559 ts_tablet_manager.cc:1905] T c4a7d334176e4e748d0700ec8b543b73 P 153e73b493bf47509fd161e335705439: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:12:08.557940 27509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:08.532687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44022 (local address 127.28.154.129:41979)
0811 09:12:08.532889 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0811 09:12:08.532895 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:12:08.532915 (+ 20us) server_negotiation.cc:407] Connection header received
0811 09:12:08.532978 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:12:08.532984 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:12:08.533065 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:12:08.533167 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:12:08.535763 (+ 2596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:08.537266 (+ 1503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:08.539422 (+ 2156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:08.540177 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:08.540298 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:12:08.544961 (+ 4663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:12:08.544974 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:12:08.544990 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:12:08.545028 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:12:08.552805 (+ 7777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:12:08.553821 (+ 1016us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:12:08.553830 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:12:08.553834 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:12:08.554052 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:12:08.554787 (+ 735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:12:08.554791 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:12:08.554794 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:12:08.555502 (+ 708us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:12:08.557105 (+ 1603us) server_negotiation.cc:1035] Waiting for connection context
0811 09:12:08.557281 (+ 176us) server_negotiation.cc:299] Negotiation successful
0811 09:12:08.557551 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 09:12:08.558861 27030 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:12:08.559027 26692 catalog_manager.cc:4841] TS 82c897eb04564e7d8870a909b3082d02 (127.28.154.131:38565): tablet c4a7d334176e4e748d0700ec8b543b73 (table test-table [id=2c96dc792abb4509b7308084bdb304e6]) successfully deleted
I20260811 09:12:08.560120 27561 ts_tablet_manager.cc:1918] T c4a7d334176e4e748d0700ec8b543b73 P 84334312b3254576991f3f49ecc134b1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:12:08.560324 27561 log.cc:1196] T c4a7d334176e4e748d0700ec8b543b73 P 84334312b3254576991f3f49ecc134b1: Deleting WAL directory at /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-1/wal/wals/c4a7d334176e4e748d0700ec8b543b73
I20260811 09:12:08.561118 27561 ts_tablet_manager.cc:1939] T c4a7d334176e4e748d0700ec8b543b73 P 84334312b3254576991f3f49ecc134b1: Deleting consensus metadata
I20260811 09:12:08.563174 26690 catalog_manager.cc:4841] TS 84334312b3254576991f3f49ecc134b1 (127.28.154.130:37427): tablet c4a7d334176e4e748d0700ec8b543b73 (table test-table [id=2c96dc792abb4509b7308084bdb304e6]) successfully deleted
I20260811 09:12:08.661690 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 26777
W20260811 09:12:08.687842 26690 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.28.154.129:41979 (error 108)
I20260811 09:12:08.688493 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 27045
I20260811 09:12:08.689972 27511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:08.689531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.129:41979 (local address 127.0.0.1:44038)
0811 09:12:08.689690 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0811 09:12:08.689805 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.129:41979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260811 09:12:08.690219 26690 catalog_manager.cc:4584] TS 153e73b493bf47509fd161e335705439 (127.28.154.129:41979): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c4a7d334176e4e748d0700ec8b543b73: Network error: Client connection negotiation failed: client connection to 127.28.154.129:41979: connect: Connection refused (error 111)
I20260811 09:12:08.707914 27510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:08.689247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54526 (local address 127.28.154.130:37427)
0811 09:12:08.689423 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 09:12:08.689429 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:12:08.689451 (+ 22us) server_negotiation.cc:407] Connection header received
0811 09:12:08.689564 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:12:08.689569 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:12:08.689655 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:12:08.689756 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:12:08.691156 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:08.692032 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:08.693380 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:08.693896 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:08.693992 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:12:08.700460 (+ 6468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:12:08.700475 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:12:08.700492 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:12:08.700544 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:12:08.704732 (+ 4188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:12:08.705581 (+ 849us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:12:08.705593 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:12:08.705597 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:12:08.705760 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:12:08.706158 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:12:08.706164 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:12:08.706167 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:12:08.706721 (+ 554us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:12:08.707022 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0811 09:12:08.707321 (+ 299us) server_negotiation.cc:299] Negotiation successful
0811 09:12:08.707558 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 09:12:08.708689 27267 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:12:08.743880 27511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:08.743373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.129:41979 (local address 127.0.0.1:44040)
0811 09:12:08.743617 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0811 09:12:08.743725 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.129:41979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
I20260811 09:12:08.744956 27511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:08.744620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.129:41979 (local address 127.0.0.1:44056)
0811 09:12:08.744769 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0811 09:12:08.744844 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.129:41979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260811 09:12:08.804523 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 27045
I20260811 09:12:08.806303 27511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:08.805860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.129:41979 (local address 127.0.0.1:44064)
0811 09:12:08.806075 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0811 09:12:08.806162 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.129:41979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260811 09:12:08.807262 27511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:08.806920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.129:41979 (local address 127.0.0.1:44076)
0811 09:12:08.807079 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0811 09:12:08.807149 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.129:41979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260811 09:12:08.831526 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 27282
I20260811 09:12:08.845593 27506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:08.832263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51264 (local address 127.28.154.131:38565)
0811 09:12:08.832481 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0811 09:12:08.832488 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:12:08.832535 (+ 47us) server_negotiation.cc:407] Connection header received
0811 09:12:08.832751 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:12:08.832757 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:12:08.832838 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:12:08.832925 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:12:08.834236 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:08.834972 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:08.836189 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:08.836725 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:08.836839 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:12:08.839544 (+ 2705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:12:08.839559 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:12:08.839579 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:12:08.839614 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:12:08.842515 (+ 2901us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:12:08.843330 (+ 815us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:12:08.843339 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:12:08.843342 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:12:08.843504 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:12:08.843891 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:12:08.843896 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:12:08.843899 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:12:08.844282 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:12:08.844607 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0811 09:12:08.845012 (+ 405us) server_negotiation.cc:299] Negotiation successful
0811 09:12:08.845245 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260811 09:12:08.846257 27480 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:12:08.887800 27511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:08.887298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.129:41979 (local address 127.0.0.1:44086)
0811 09:12:08.887516 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0811 09:12:08.887617 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.129:41979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260811 09:12:08.888856 27511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:08.888452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.129:41979 (local address 127.0.0.1:44102)
0811 09:12:08.888665 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0811 09:12:08.888741 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.129:41979: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260811 09:12:08.937395 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 27282
I20260811 09:12:08.964228 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 26652
I20260811 09:12:08.965759 26766 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:12:09.084528 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 26652
I20260811 09:12:29.116281 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.154.190:40403
--webserver_interface=127.28.154.190
--webserver_port=44341
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab
--principal=kudu/127.28.154.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:38289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.154.190:40403
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:12:29.483024 27573 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:12:29.483348 27573 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:12:29.483450 27573 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:12:29.491127 27573 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 09:12:29.491257 27573 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 09:12:29.491298 27573 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 09:12:29.491362 27573 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:12:29.491429 27573 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:12:29.491506 27573 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:12:29.491554 27573 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:12:29.491606 27573 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:12:29.502115 27573 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:38289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.28.154.190:40403
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab
--principal=kudu/127.28.154.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.154.190:40403
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.154.190
--webserver_port=44341
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:12:29.503455 27573 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:12:29.504799 27573 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:12:29.517091 27583 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:12:29.517145 27585 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:12:29.517061 27588 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:12:29.517459 27573 server_base.cc:1034] running on GCE node
I20260811 09:12:29.518453 27573 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:12:29.519915 27573 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:12:29.521163 27573 hybrid_clock.cc:648] HybridClock initialized: now 1786439549521082 us; error 53 us; skew 500 ppm
I20260811 09:12:29.521531 27573 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:12:29 dist-test-slave-xt4k krb5kdc[25585](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439549, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:12:29.524938 27573 init.cc:377] Logged in from keytab as kudu/127.28.154.190@KRBTEST.COM (short username kudu)
I20260811 09:12:29.526877 27573 webserver.cc:460] Webserver started at http://127.28.154.190:44341/ using document root <none> and password file <none>
I20260811 09:12:29.527678 27573 fs_manager.cc:359] Metadata directory not provided
I20260811 09:12:29.527802 27573 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:12:29.533753 27573 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.002s
I20260811 09:12:29.536895 27601 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:12:29.538333 27573 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 09:12:29.538508 27573 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/master-0/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/master-0/wal
uuid: "b740e22f6c334fb38cdc579e01606268"
format_stamp: "Formatted at 2026-08-11 09:11:43 on dist-test-slave-xt4k"
server_key: "ada7b95799df2469c5c6b10b1a71d5d8"
server_key_iv: "76280d86cd4f431e0dd06f2758862612"
server_key_version: "encryptionkey@0"
I20260811 09:12:29.539510 27573 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:12:29.570394 27573 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:12:29.574813 27573 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:12:29.575093 27573 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:12:29.593577 27573 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.190:40403
I20260811 09:12:29.593600 27682 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.190:40403 every 8 connection(s)
I20260811 09:12:29.595083 27573 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
I20260811 09:12:29.599306 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 27573
I20260811 09:12:29.600172 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.154.129:41979
--local_ip_for_outbound_sockets=127.28.154.129
--tserver_master_addrs=127.28.154.190:40403
--webserver_port=36297
--webserver_interface=127.28.154.129
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.129.keytab
--principal=kudu/127.28.154.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:38289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 09:12:29.602135 27683 sys_catalog.cc:263] Verifying existing consensus state
I20260811 09:12:29.605679 27683 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b740e22f6c334fb38cdc579e01606268: Bootstrap starting.
I20260811 09:12:29.630550 27683 log.cc:824] T 00000000000000000000000000000000 P b740e22f6c334fb38cdc579e01606268: Log is configured to *not* fsync() on all Append() calls
I20260811 09:12:29.653127 27683 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b740e22f6c334fb38cdc579e01606268: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 09:12:29.653831 27683 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b740e22f6c334fb38cdc579e01606268: Bootstrap complete.
I20260811 09:12:29.667748 27683 raft_consensus.cc:348] T 00000000000000000000000000000000 P b740e22f6c334fb38cdc579e01606268 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b740e22f6c334fb38cdc579e01606268" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 40403 } }
I20260811 09:12:29.669114 27683 raft_consensus.cc:729] T 00000000000000000000000000000000 P b740e22f6c334fb38cdc579e01606268 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: b740e22f6c334fb38cdc579e01606268, State: Initialized, Role: FOLLOWER
I20260811 09:12:29.669915 27683 consensus_queue.cc:260] T 00000000000000000000000000000000 P b740e22f6c334fb38cdc579e01606268 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b740e22f6c334fb38cdc579e01606268" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 40403 } }
I20260811 09:12:29.670245 27683 raft_consensus.cc:388] T 00000000000000000000000000000000 P b740e22f6c334fb38cdc579e01606268 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:12:29.670394 27683 raft_consensus.cc:482] T 00000000000000000000000000000000 P b740e22f6c334fb38cdc579e01606268 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:12:29.670593 27683 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b740e22f6c334fb38cdc579e01606268 [term 2 FOLLOWER]: Advancing to term 3
I20260811 09:12:29.676594 27683 raft_consensus.cc:504] T 00000000000000000000000000000000 P b740e22f6c334fb38cdc579e01606268 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b740e22f6c334fb38cdc579e01606268" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 40403 } }
I20260811 09:12:29.677350 27683 leader_election.cc:302] T 00000000000000000000000000000000 P b740e22f6c334fb38cdc579e01606268 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b740e22f6c334fb38cdc579e01606268; no voters:
I20260811 09:12:29.678158 27683 leader_election.cc:288] T 00000000000000000000000000000000 P b740e22f6c334fb38cdc579e01606268 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260811 09:12:29.678306 27690 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b740e22f6c334fb38cdc579e01606268 [term 3 FOLLOWER]: Leader election won for term 3
I20260811 09:12:29.679595 27690 raft_consensus.cc:686] T 00000000000000000000000000000000 P b740e22f6c334fb38cdc579e01606268 [term 3 LEADER]: Becoming Leader. State: Replica: b740e22f6c334fb38cdc579e01606268, State: Running, Role: LEADER
I20260811 09:12:29.680217 27690 consensus_queue.cc:237] T 00000000000000000000000000000000 P b740e22f6c334fb38cdc579e01606268 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b740e22f6c334fb38cdc579e01606268" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 40403 } }
I20260811 09:12:29.680989 27683 sys_catalog.cc:564] T 00000000000000000000000000000000 P b740e22f6c334fb38cdc579e01606268 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:12:29.682602 27692 sys_catalog.cc:455] T 00000000000000000000000000000000 P b740e22f6c334fb38cdc579e01606268 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b740e22f6c334fb38cdc579e01606268. Latest consensus state: current_term: 3 leader_uuid: "b740e22f6c334fb38cdc579e01606268" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b740e22f6c334fb38cdc579e01606268" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 40403 } } }
I20260811 09:12:29.683001 27692 sys_catalog.cc:458] T 00000000000000000000000000000000 P b740e22f6c334fb38cdc579e01606268 [sys.catalog]: This master's current role is: LEADER
I20260811 09:12:29.683560 27691 sys_catalog.cc:455] T 00000000000000000000000000000000 P b740e22f6c334fb38cdc579e01606268 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "b740e22f6c334fb38cdc579e01606268" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b740e22f6c334fb38cdc579e01606268" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 40403 } } }
I20260811 09:12:29.683853 27691 sys_catalog.cc:458] T 00000000000000000000000000000000 P b740e22f6c334fb38cdc579e01606268 [sys.catalog]: This master's current role is: LEADER
I20260811 09:12:29.692456 27701 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:12:29.701378 27701 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:12:29.703051 27701 catalog_manager.cc:1259] Loaded cluster ID: 959c088f5a144412a35c0e95cbab5cb2
I20260811 09:12:29.703235 27701 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:12:29.707521 27701 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:12:29.709360 27701 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b740e22f6c334fb38cdc579e01606268: Loaded TSK: 0
I20260811 09:12:29.709731 27701 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b740e22f6c334fb38cdc579e01606268: Loaded TSK: 1
F20260811 09:12:29.710959 27701 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f51759769ad google::LogMessage::Fail() at ??:0
@ 0x7f517597988a google::LogMessage::SendToLog() at ??:0
@ 0x7f51759764de google::LogMessage::Flush() at ??:0
@ 0x7f5175977a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f51796da7d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f51796c973e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f518b3a51c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f518b3a4783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f518b4302a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7f518b4300c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7f518b47a5e9 std::function<>::operator()() at ??:0
@ 0x7f518b3a8cdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f518b3a6c36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f518b42e610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f518b42e439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f51785c2345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f51785c591c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f51785c5799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f51785a0eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f517b87a6db start_thread at ??:0
@ 0x7f517481061f clone at ??:0
W20260811 09:12:30.067123 27685 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:12:30.067436 27685 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:12:30.067544 27685 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:12:30.075469 27685 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:12:30.075614 27685 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:12:30.075735 27685 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.154.129
I20260811 09:12:30.086437 27685 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:38289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.129.keytab
--principal=kudu/127.28.154.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.154.129:41979
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.154.129
--webserver_port=36297
--tserver_master_addrs=127.28.154.190:40403
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.154.129
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:12:30.087989 27685 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:12:30.089504 27685 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:12:30.103618 27774 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:12:30.103652 27780 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:12:30.103596 27775 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:12:30.104730 27685 server_base.cc:1034] running on GCE node
I20260811 09:12:30.105373 27685 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:12:30.106386 27685 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:12:30.107633 27685 hybrid_clock.cc:648] HybridClock initialized: now 1786439550107582 us; error 38 us; skew 500 ppm
I20260811 09:12:30.107980 27685 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:12:30 dist-test-slave-xt4k krb5kdc[25585](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439550, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:12:30.111784 27685 init.cc:377] Logged in from keytab as kudu/127.28.154.129@KRBTEST.COM (short username kudu)
I20260811 09:12:30.113757 27685 webserver.cc:460] Webserver started at http://127.28.154.129:36297/ using document root <none> and password file <none>
I20260811 09:12:30.114728 27685 fs_manager.cc:359] Metadata directory not provided
I20260811 09:12:30.114878 27685 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:12:30.122992 27685 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.001s sys 0.004s
I20260811 09:12:30.126622 27793 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:12:30.128160 27685 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 09:12:30.128387 27685 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-0/wal
uuid: "153e73b493bf47509fd161e335705439"
format_stamp: "Formatted at 2026-08-11 09:11:43 on dist-test-slave-xt4k"
server_key: "a08bdbd47d5f29477d745c6e9f160c9d"
server_key_iv: "ae5122a0f882baf48ec9c9e6bd0c8b01"
server_key_version: "encryptionkey@0"
I20260811 09:12:30.129457 27685 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:12:30.156519 27685 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:12:30.161185 27685 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:12:30.161473 27685 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:12:30.162588 27685 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:12:30.164683 27808 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260811 09:12:30.170806 27685 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 09:12:30.170965 27685 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.007s user 0.000s sys 0.001s
I20260811 09:12:30.171106 27685 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 09:12:30.174551 27685 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 09:12:30.174646 27685 ts_tablet_manager.cc:589] Time spent register tablets: real 0.004s user 0.000s sys 0.001s
I20260811 09:12:30.175146 27808 tablet_bootstrap.cc:492] T c4a7d334176e4e748d0700ec8b543b73 P 153e73b493bf47509fd161e335705439: Bootstrap starting.
I20260811 09:12:30.184523 27808 log.cc:824] T c4a7d334176e4e748d0700ec8b543b73 P 153e73b493bf47509fd161e335705439: Log is configured to *not* fsync() on all Append() calls
I20260811 09:12:30.201184 27808 tablet_bootstrap.cc:492] T c4a7d334176e4e748d0700ec8b543b73 P 153e73b493bf47509fd161e335705439: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 09:12:30.202018 27808 tablet_bootstrap.cc:492] T c4a7d334176e4e748d0700ec8b543b73 P 153e73b493bf47509fd161e335705439: Bootstrap complete.
I20260811 09:12:30.202570 27808 ts_tablet_manager.cc:1397] T c4a7d334176e4e748d0700ec8b543b73 P 153e73b493bf47509fd161e335705439: Time spent bootstrapping tablet: real 0.028s user 0.013s sys 0.012s
I20260811 09:12:30.212307 27808 raft_consensus.cc:348] T c4a7d334176e4e748d0700ec8b543b73 P 153e73b493bf47509fd161e335705439 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82c897eb04564e7d8870a909b3082d02" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 38565 } } peers { permanent_uuid: "153e73b493bf47509fd161e335705439" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41979 } } peers { permanent_uuid: "84334312b3254576991f3f49ecc134b1" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 37427 } }
I20260811 09:12:30.213995 27808 raft_consensus.cc:729] T c4a7d334176e4e748d0700ec8b543b73 P 153e73b493bf47509fd161e335705439 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 153e73b493bf47509fd161e335705439, State: Initialized, Role: FOLLOWER
I20260811 09:12:30.215001 27808 consensus_queue.cc:260] T c4a7d334176e4e748d0700ec8b543b73 P 153e73b493bf47509fd161e335705439 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82c897eb04564e7d8870a909b3082d02" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 38565 } } peers { permanent_uuid: "153e73b493bf47509fd161e335705439" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41979 } } peers { permanent_uuid: "84334312b3254576991f3f49ecc134b1" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 37427 } }
I20260811 09:12:30.216799 27808 ts_tablet_manager.cc:1428] T c4a7d334176e4e748d0700ec8b543b73 P 153e73b493bf47509fd161e335705439: Time spent starting tablet: real 0.014s user 0.006s sys 0.009s
I20260811 09:12:30.218914 27685 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.129:41979
I20260811 09:12:30.218974 27994 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.129:41979 every 8 connection(s)
I20260811 09:12:30.220163 27685 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-0/data/info.pb
I20260811 09:12:30.223132 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 27685
I20260811 09:12:30.223946 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.154.130:37427
--local_ip_for_outbound_sockets=127.28.154.130
--tserver_master_addrs=127.28.154.190:40403
--webserver_port=45323
--webserver_interface=127.28.154.130
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.130.keytab
--principal=kudu/127.28.154.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:38289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 09:12:30.225397 27998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:30.223218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:40403 (local address 127.28.154.129:56929)
0811 09:12:30.224202 (+ 984us) negotiation.cc:107] Waiting for socket to connect
0811 09:12:30.224531 (+ 329us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:40403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":653,"thread_start_us":420,"threads_started":1}
W20260811 09:12:30.226042 27995 heartbeater.cc:644] Failed to heartbeat to 127.28.154.190:40403 (0 consecutive failures): Network error: Failed to ping master at 127.28.154.190:40403: Client connection negotiation failed: client connection to 127.28.154.190:40403: connect: Connection refused (error 111)
I20260811 09:12:30.226890 27998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:30.226568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:40403 (local address 127.28.154.129:49813)
0811 09:12:30.226719 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0811 09:12:30.226788 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:40403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260811 09:12:30.227741 27998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:30.227401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:40403 (local address 127.28.154.129:41467)
0811 09:12:30.227554 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0811 09:12:30.227625 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:40403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260811 09:12:30.227968 27995 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 09:12:30.576731 28000 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:12:30.577081 28000 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:12:30.577210 28000 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:12:30.585075 28000 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:12:30.585212 28000 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:12:30.585371 28000 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.154.130
I20260811 09:12:30.595894 28000 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:38289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.130.keytab
--principal=kudu/127.28.154.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.154.130:37427
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.154.130
--webserver_port=45323
--tserver_master_addrs=127.28.154.190:40403
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.154.130
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:12:30.597280 28000 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:12:30.598645 28000 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:12:30.612166 28009 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:12:30.612174 28010 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:12:30.612170 28016 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:12:30.612834 28000 server_base.cc:1034] running on GCE node
I20260811 09:12:30.613518 28000 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:12:30.614552 28000 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:12:30.615787 28000 hybrid_clock.cc:648] HybridClock initialized: now 1786439550615735 us; error 38 us; skew 500 ppm
I20260811 09:12:30.616130 28000 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:12:30 dist-test-slave-xt4k krb5kdc[25585](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439550, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:12:30.619343 28000 init.cc:377] Logged in from keytab as kudu/127.28.154.130@KRBTEST.COM (short username kudu)
I20260811 09:12:30.620898 28000 webserver.cc:460] Webserver started at http://127.28.154.130:45323/ using document root <none> and password file <none>
I20260811 09:12:30.621587 28000 fs_manager.cc:359] Metadata directory not provided
I20260811 09:12:30.621711 28000 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:12:30.627537 28000 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 09:12:30.630678 28029 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:12:30.632066 28000 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 09:12:30.632239 28000 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-1/wal
uuid: "84334312b3254576991f3f49ecc134b1"
format_stamp: "Formatted at 2026-08-11 09:11:44 on dist-test-slave-xt4k"
server_key: "57769100a6c65364fdc9ec81b45f0a3a"
server_key_iv: "eff99c23b261da5a2ed4654f64797ee0"
server_key_version: "encryptionkey@0"
I20260811 09:12:30.633378 28000 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:12:30.661072 28000 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:12:30.665902 28000 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:12:30.666188 28000 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:12:30.667316 28000 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:12:30.668787 28000 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:12:30.668879 28000 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:12:30.668980 28000 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:12:30.669050 28000 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:12:30.699390 28000 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.130:37427
I20260811 09:12:30.699452 28228 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.130:37427 every 8 connection(s)
I20260811 09:12:30.700918 28000 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-1/data/info.pb
I20260811 09:12:30.705602 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 28000
I20260811 09:12:30.705952 28231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:30.704015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:40403 (local address 127.28.154.130:34327)
0811 09:12:30.704804 (+ 789us) negotiation.cc:107] Waiting for socket to connect
0811 09:12:30.705065 (+ 261us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:40403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":531,"thread_start_us":322,"threads_started":1}
I20260811 09:12:30.706264 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.154.131:38565
--local_ip_for_outbound_sockets=127.28.154.131
--tserver_master_addrs=127.28.154.190:40403
--webserver_port=43089
--webserver_interface=127.28.154.131
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.131.keytab
--principal=kudu/127.28.154.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:38289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:12:30.706614 28229 heartbeater.cc:644] Failed to heartbeat to 127.28.154.190:40403 (0 consecutive failures): Network error: Failed to ping master at 127.28.154.190:40403: Client connection negotiation failed: client connection to 127.28.154.190:40403: connect: Connection refused (error 111)
I20260811 09:12:30.707453 28231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:30.707130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:40403 (local address 127.28.154.130:55067)
0811 09:12:30.707275 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0811 09:12:30.707344 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:40403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260811 09:12:30.708256 28231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:30.707941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:40403 (local address 127.28.154.130:42929)
0811 09:12:30.708079 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0811 09:12:30.708151 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:40403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260811 09:12:30.708508 28229 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 09:12:31.045998 28232 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:12:31.046352 28232 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:12:31.046746 28232 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:12:31.055207 28232 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:12:31.055372 28232 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:12:31.055550 28232 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.154.131
I20260811 09:12:31.066198 28232 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:38289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.131.keytab
--principal=kudu/127.28.154.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.154.131:38565
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.154.131
--webserver_port=43089
--tserver_master_addrs=127.28.154.190:40403
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.154.131
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:12:31.067589 28232 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:12:31.068976 28232 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:12:31.082370 28243 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:12:31.082314 28241 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:12:31.082458 28252 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:12:31.083069 28232 server_base.cc:1034] running on GCE node
I20260811 09:12:31.083750 28232 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:12:31.084776 28232 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:12:31.086005 28232 hybrid_clock.cc:648] HybridClock initialized: now 1786439551085934 us; error 43 us; skew 500 ppm
I20260811 09:12:31.086342 28232 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:12:31 dist-test-slave-xt4k krb5kdc[25585](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439551, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:12:31.089545 28232 init.cc:377] Logged in from keytab as kudu/127.28.154.131@KRBTEST.COM (short username kudu)
I20260811 09:12:31.091288 28232 webserver.cc:460] Webserver started at http://127.28.154.131:43089/ using document root <none> and password file <none>
I20260811 09:12:31.092067 28232 fs_manager.cc:359] Metadata directory not provided
I20260811 09:12:31.092185 28232 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:12:31.098512 28232 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 09:12:31.101804 28265 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:12:31.103668 28232 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 09:12:31.103864 28232 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-2/wal
uuid: "82c897eb04564e7d8870a909b3082d02"
format_stamp: "Formatted at 2026-08-11 09:11:45 on dist-test-slave-xt4k"
server_key: "a90f2eeafb5bab0abe030a448def00db"
server_key_iv: "6895a616d1b1d17f28b8186124151e9f"
server_key_version: "encryptionkey@0"
I20260811 09:12:31.104970 28232 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:12:31.120585 28232 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:12:31.125481 28232 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:12:31.125764 28232 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:12:31.126780 28232 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:12:31.128149 28232 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:12:31.128259 28232 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:12:31.128386 28232 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:12:31.128460 28232 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:12:31.160506 28232 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.131:38565
I20260811 09:12:31.160558 28459 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.131:38565 every 8 connection(s)
I20260811 09:12:31.161940 28232 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786439250192320-29290-0/minicluster-data/ts-2/data/info.pb
I20260811 09:12:31.166062 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 28232
I20260811 09:12:31.167078 28465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:31.164983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:40403 (local address 127.28.154.131:38007)
0811 09:12:31.165745 (+ 762us) negotiation.cc:107] Waiting for socket to connect
0811 09:12:31.165984 (+ 239us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:40403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":276,"threads_started":1}
W20260811 09:12:31.167768 28463 heartbeater.cc:644] Failed to heartbeat to 127.28.154.190:40403 (0 consecutive failures): Network error: Failed to ping master at 127.28.154.190:40403: Client connection negotiation failed: client connection to 127.28.154.190:40403: connect: Connection refused (error 111)
I20260811 09:12:31.168764 28465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:31.168390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:40403 (local address 127.28.154.131:52399)
0811 09:12:31.168563 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0811 09:12:31.168634 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:40403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260811 09:12:31.169677 28465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:31.169302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:40403 (local address 127.28.154.131:50461)
0811 09:12:31.169485 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0811 09:12:31.169566 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:40403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
W20260811 09:12:31.169983 28463 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 09:12:31.229465 28471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:31.228612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:40403 (local address 127.28.154.129:46077)
0811 09:12:31.229146 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0811 09:12:31.229244 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:40403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":288,"threads_started":1}
I20260811 09:12:31.531728 28473 raft_consensus.cc:482] T c4a7d334176e4e748d0700ec8b543b73 P 153e73b493bf47509fd161e335705439 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:12:31.532049 28473 raft_consensus.cc:504] T c4a7d334176e4e748d0700ec8b543b73 P 153e73b493bf47509fd161e335705439 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82c897eb04564e7d8870a909b3082d02" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 38565 } } peers { permanent_uuid: "153e73b493bf47509fd161e335705439" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41979 } } peers { permanent_uuid: "84334312b3254576991f3f49ecc134b1" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 37427 } }
I20260811 09:12:31.533947 28473 leader_election.cc:288] T c4a7d334176e4e748d0700ec8b543b73 P 153e73b493bf47509fd161e335705439 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 82c897eb04564e7d8870a909b3082d02 (127.28.154.131:38565), 84334312b3254576991f3f49ecc134b1 (127.28.154.130:37427)
Aug 11 09:12:31 dist-test-slave-xt4k krb5kdc[25585](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439550, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.129@KRBTEST.COM for kudu/127.28.154.130@KRBTEST.COM
Aug 11 09:12:31 dist-test-slave-xt4k krb5kdc[25585](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439550, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.129@KRBTEST.COM for kudu/127.28.154.131@KRBTEST.COM
I20260811 09:12:31.556037 28475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:31.534634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.129:36351 (local address 127.28.154.130:37427)
0811 09:12:31.536305 (+ 1671us) server_negotiation.cc:206] Beginning negotiation
0811 09:12:31.536320 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 09:12:31.536922 (+ 602us) server_negotiation.cc:407] Connection header received
0811 09:12:31.538696 (+ 1774us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:12:31.538709 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:12:31.539078 (+ 369us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:12:31.539437 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:12:31.541457 (+ 2020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:31.542642 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:31.544421 (+ 1779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:31.545065 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:31.545243 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:12:31.549036 (+ 3793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:12:31.549046 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:12:31.549063 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:12:31.549107 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:12:31.552261 (+ 3154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:12:31.553108 (+ 847us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:12:31.553115 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:12:31.553120 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:12:31.553281 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:12:31.553729 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:12:31.553732 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:12:31.553734 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:12:31.554253 (+ 519us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:12:31.554436 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0811 09:12:31.555264 (+ 828us) server_negotiation.cc:299] Negotiation successful
0811 09:12:31.555599 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1485,"thread_start_us":1358,"threads_started":1}
I20260811 09:12:31.556056 28471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:31.534431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.131:38565 (local address 127.28.154.129:55001)
0811 09:12:31.534815 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0811 09:12:31.534866 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 09:12:31.535533 (+ 667us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:12:31.539540 (+ 4007us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:12:31.539551 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:12:31.540499 (+ 948us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:12:31.541254 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:12:31.541276 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:12:31.542828 (+ 1552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:12:31.542832 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:12:31.544255 (+ 1423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:12:31.544269 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:12:31.545259 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:12:31.545266 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:12:31.545537 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:12:31.546252 (+ 715us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:12:31.546275 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:12:31.548878 (+ 2603us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:12:31.552541 (+ 3663us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:12:31.552546 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:12:31.552550 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:12:31.553003 (+ 453us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:12:31.553434 (+ 431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:12:31.553439 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:12:31.553447 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:12:31.553615 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:12:31.554415 (+ 800us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:12:31.554420 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:12:31.554728 (+ 308us) client_negotiation.cc:769] Sending connection context
0811 09:12:31.554999 (+ 271us) client_negotiation.cc:240] Negotiation successful
0811 09:12:31.555592 (+ 593us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"mutex_wait_us":196,"spinlock_wait_cycles":18560}
I20260811 09:12:31.556049 28478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:31.534599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.129:55001 (local address 127.28.154.131:38565)
0811 09:12:31.536869 (+ 2270us) server_negotiation.cc:206] Beginning negotiation
0811 09:12:31.536882 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 09:12:31.536923 (+ 41us) server_negotiation.cc:407] Connection header received
0811 09:12:31.538499 (+ 1576us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:12:31.538507 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:12:31.538951 (+ 444us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:12:31.539905 (+ 954us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:12:31.541450 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:31.542666 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:31.544415 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:31.545046 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:31.545218 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:12:31.549143 (+ 3925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:12:31.549151 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:12:31.549164 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:12:31.549201 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:12:31.552417 (+ 3216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:12:31.553151 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:12:31.553158 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:12:31.553164 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:12:31.553321 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:12:31.553800 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:12:31.553804 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:12:31.553806 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:12:31.554285 (+ 479us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:12:31.554464 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0811 09:12:31.555260 (+ 796us) server_negotiation.cc:299] Negotiation successful
0811 09:12:31.555648 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2093,"thread_start_us":1881,"threads_started":1}
I20260811 09:12:31.556313 28474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:31.534556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.130:37427 (local address 127.28.154.129:36351)
0811 09:12:31.536757 (+ 2201us) negotiation.cc:107] Waiting for socket to connect
0811 09:12:31.536777 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 09:12:31.536924 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:12:31.539540 (+ 2616us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:12:31.539551 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:12:31.540497 (+ 946us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:12:31.541253 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:12:31.541275 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:12:31.542836 (+ 1561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:12:31.542840 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:12:31.544252 (+ 1412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:12:31.544265 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:12:31.545306 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:12:31.545315 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:12:31.545534 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:12:31.546304 (+ 770us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:12:31.546321 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:12:31.548704 (+ 2383us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:12:31.552425 (+ 3721us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:12:31.552435 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:12:31.552453 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:12:31.552968 (+ 515us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:12:31.553392 (+ 424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:12:31.553397 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:12:31.553401 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:12:31.553615 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:12:31.554395 (+ 780us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:12:31.554404 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:12:31.554727 (+ 323us) client_negotiation.cc:769] Sending connection context
0811 09:12:31.554999 (+ 272us) client_negotiation.cc:240] Negotiation successful
0811 09:12:31.555751 (+ 752us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2010,"mutex_wait_us":363,"spinlock_wait_cycles":41728,"thread_start_us":1787,"threads_started":1}
W20260811 09:12:31.559296 27794 leader_election.cc:341] T c4a7d334176e4e748d0700ec8b543b73 P 153e73b493bf47509fd161e335705439 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 84334312b3254576991f3f49ecc134b1 (127.28.154.130:37427): Not found: Tablet not found: c4a7d334176e4e748d0700ec8b543b73
W20260811 09:12:31.559558 27796 leader_election.cc:341] T c4a7d334176e4e748d0700ec8b543b73 P 153e73b493bf47509fd161e335705439 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 82c897eb04564e7d8870a909b3082d02 (127.28.154.131:38565): Not found: Tablet not found: c4a7d334176e4e748d0700ec8b543b73
I20260811 09:12:31.559758 27796 leader_election.cc:302] T c4a7d334176e4e748d0700ec8b543b73 P 153e73b493bf47509fd161e335705439 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 153e73b493bf47509fd161e335705439; no voters: 82c897eb04564e7d8870a909b3082d02, 84334312b3254576991f3f49ecc134b1
I20260811 09:12:31.560159 28473 raft_consensus.cc:2726] T c4a7d334176e4e748d0700ec8b543b73 P 153e73b493bf47509fd161e335705439 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 09:12:31.710067 28479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:31.709104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:40403 (local address 127.28.154.130:60759)
0811 09:12:31.709667 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0811 09:12:31.709812 (+ 145us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:40403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":274,"threads_started":1}
I20260811 09:12:32.171442 28480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:32.170612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:40403 (local address 127.28.154.131:36243)
0811 09:12:32.171108 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0811 09:12:32.171234 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:40403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":244,"threads_started":1}
I20260811 09:12:32.231130 28481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:32.230442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:40403 (local address 127.28.154.129:54255)
0811 09:12:32.230864 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0811 09:12:32.230953 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:40403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":210,"threads_started":1}
I20260811 09:12:32.711825 28482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:32.710969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:40403 (local address 127.28.154.130:40507)
0811 09:12:32.711518 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0811 09:12:32.711642 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:40403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":261,"threads_started":1}
I20260811 09:12:33.173481 28483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:33.172586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:40403 (local address 127.28.154.131:41913)
0811 09:12:33.173131 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0811 09:12:33.173255 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:40403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":260,"threads_started":1}
I20260811 09:12:33.233630 28484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:33.232424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:40403 (local address 127.28.154.129:51843)
0811 09:12:33.233299 (+ 875us) negotiation.cc:107] Waiting for socket to connect
0811 09:12:33.233427 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:40403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":713,"thread_start_us":524,"threads_started":1}
I20260811 09:12:33.713819 28485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:33.713012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:40403 (local address 127.28.154.130:50927)
0811 09:12:33.713547 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0811 09:12:33.713643 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:40403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":244,"threads_started":1}
I20260811 09:12:33.751235 28487 raft_consensus.cc:482] T c4a7d334176e4e748d0700ec8b543b73 P 153e73b493bf47509fd161e335705439 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:12:33.751443 28487 raft_consensus.cc:504] T c4a7d334176e4e748d0700ec8b543b73 P 153e73b493bf47509fd161e335705439 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82c897eb04564e7d8870a909b3082d02" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 38565 } } peers { permanent_uuid: "153e73b493bf47509fd161e335705439" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41979 } } peers { permanent_uuid: "84334312b3254576991f3f49ecc134b1" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 37427 } }
I20260811 09:12:33.752144 28487 leader_election.cc:288] T c4a7d334176e4e748d0700ec8b543b73 P 153e73b493bf47509fd161e335705439 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 82c897eb04564e7d8870a909b3082d02 (127.28.154.131:38565), 84334312b3254576991f3f49ecc134b1 (127.28.154.130:37427)
W20260811 09:12:33.753211 27794 leader_election.cc:341] T c4a7d334176e4e748d0700ec8b543b73 P 153e73b493bf47509fd161e335705439 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 84334312b3254576991f3f49ecc134b1 (127.28.154.130:37427): Not found: Tablet not found: c4a7d334176e4e748d0700ec8b543b73
W20260811 09:12:33.753382 27796 leader_election.cc:341] T c4a7d334176e4e748d0700ec8b543b73 P 153e73b493bf47509fd161e335705439 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 82c897eb04564e7d8870a909b3082d02 (127.28.154.131:38565): Not found: Tablet not found: c4a7d334176e4e748d0700ec8b543b73
I20260811 09:12:33.753502 27796 leader_election.cc:302] T c4a7d334176e4e748d0700ec8b543b73 P 153e73b493bf47509fd161e335705439 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 153e73b493bf47509fd161e335705439; no voters: 82c897eb04564e7d8870a909b3082d02, 84334312b3254576991f3f49ecc134b1
I20260811 09:12:33.753762 28487 raft_consensus.cc:2726] T c4a7d334176e4e748d0700ec8b543b73 P 153e73b493bf47509fd161e335705439 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 09:12:34.175397 28491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:34.174605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:40403 (local address 127.28.154.131:51955)
0811 09:12:34.175131 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0811 09:12:34.175228 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:40403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":415,"thread_start_us":266,"threads_started":1}
I20260811 09:12:34.235457 28495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:34.234647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:40403 (local address 127.28.154.129:59331)
0811 09:12:34.235167 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0811 09:12:34.235262 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:40403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":237,"threads_started":1}
I20260811 09:12:34.715581 28497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:34.714796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:40403 (local address 127.28.154.130:45301)
0811 09:12:34.715298 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0811 09:12:34.715384 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:40403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":273,"threads_started":1}
I20260811 09:12:35.177400 28499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:35.176465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:40403 (local address 127.28.154.131:47779)
0811 09:12:35.177036 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0811 09:12:35.177153 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:40403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":259,"threads_started":1}
I20260811 09:12:35.237427 28501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:35.236511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:40403 (local address 127.28.154.129:35859)
0811 09:12:35.237075 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0811 09:12:35.237228 (+ 153us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:40403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":254,"threads_started":1}
I20260811 09:12:35.375766 28503 raft_consensus.cc:482] T c4a7d334176e4e748d0700ec8b543b73 P 153e73b493bf47509fd161e335705439 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:12:35.376003 28503 raft_consensus.cc:504] T c4a7d334176e4e748d0700ec8b543b73 P 153e73b493bf47509fd161e335705439 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82c897eb04564e7d8870a909b3082d02" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 38565 } } peers { permanent_uuid: "153e73b493bf47509fd161e335705439" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41979 } } peers { permanent_uuid: "84334312b3254576991f3f49ecc134b1" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 37427 } }
I20260811 09:12:35.376703 28503 leader_election.cc:288] T c4a7d334176e4e748d0700ec8b543b73 P 153e73b493bf47509fd161e335705439 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 82c897eb04564e7d8870a909b3082d02 (127.28.154.131:38565), 84334312b3254576991f3f49ecc134b1 (127.28.154.130:37427)
W20260811 09:12:35.377671 27794 leader_election.cc:341] T c4a7d334176e4e748d0700ec8b543b73 P 153e73b493bf47509fd161e335705439 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 84334312b3254576991f3f49ecc134b1 (127.28.154.130:37427): Not found: Tablet not found: c4a7d334176e4e748d0700ec8b543b73
W20260811 09:12:35.377894 27796 leader_election.cc:341] T c4a7d334176e4e748d0700ec8b543b73 P 153e73b493bf47509fd161e335705439 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 82c897eb04564e7d8870a909b3082d02 (127.28.154.131:38565): Not found: Tablet not found: c4a7d334176e4e748d0700ec8b543b73
I20260811 09:12:35.378036 27796 leader_election.cc:302] T c4a7d334176e4e748d0700ec8b543b73 P 153e73b493bf47509fd161e335705439 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 153e73b493bf47509fd161e335705439; no voters: 82c897eb04564e7d8870a909b3082d02, 84334312b3254576991f3f49ecc134b1
I20260811 09:12:35.378286 28503 raft_consensus.cc:2726] T c4a7d334176e4e748d0700ec8b543b73 P 153e73b493bf47509fd161e335705439 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 09:12:35.717387 28504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:35.716633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:40403 (local address 127.28.154.130:37493)
0811 09:12:35.717123 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0811 09:12:35.717215 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:40403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":240,"threads_started":1}
I20260811 09:12:36.179353 28505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:36.178426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:40403 (local address 127.28.154.131:59621)
0811 09:12:36.179007 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0811 09:12:36.179138 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:40403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":295,"threads_started":1}
I20260811 09:12:36.239181 28507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:36.238495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:40403 (local address 127.28.154.129:43101)
0811 09:12:36.238915 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0811 09:12:36.239003 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:40403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":212,"threads_started":1}
I20260811 09:12:36.719091 28509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:36.718332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:40403 (local address 127.28.154.130:52409)
0811 09:12:36.718796 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0811 09:12:36.718884 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:40403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":237,"threads_started":1}
I20260811 09:12:37.181310 28510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:37.180394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:40403 (local address 127.28.154.131:53669)
0811 09:12:37.180981 (+ 587us) negotiation.cc:107] Waiting for socket to connect
0811 09:12:37.181094 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:40403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":325,"threads_started":1}
I20260811 09:12:37.241011 28511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:37.240091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:40403 (local address 127.28.154.129:37729)
0811 09:12:37.240596 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0811 09:12:37.240797 (+ 201us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:40403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":278,"threads_started":1}
I20260811 09:12:37.720964 28517 raft_consensus.cc:482] T c4a7d334176e4e748d0700ec8b543b73 P 153e73b493bf47509fd161e335705439 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:12:37.721108 28513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:37.720106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:40403 (local address 127.28.154.130:32941)
0811 09:12:37.720804 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0811 09:12:37.720913 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:40403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":546,"thread_start_us":326,"threads_started":1}
I20260811 09:12:37.721177 28517 raft_consensus.cc:504] T c4a7d334176e4e748d0700ec8b543b73 P 153e73b493bf47509fd161e335705439 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82c897eb04564e7d8870a909b3082d02" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 38565 } } peers { permanent_uuid: "153e73b493bf47509fd161e335705439" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41979 } } peers { permanent_uuid: "84334312b3254576991f3f49ecc134b1" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 37427 } }
I20260811 09:12:37.721868 28517 leader_election.cc:288] T c4a7d334176e4e748d0700ec8b543b73 P 153e73b493bf47509fd161e335705439 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 82c897eb04564e7d8870a909b3082d02 (127.28.154.131:38565), 84334312b3254576991f3f49ecc134b1 (127.28.154.130:37427)
W20260811 09:12:37.722770 27794 leader_election.cc:341] T c4a7d334176e4e748d0700ec8b543b73 P 153e73b493bf47509fd161e335705439 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 84334312b3254576991f3f49ecc134b1 (127.28.154.130:37427): Not found: Tablet not found: c4a7d334176e4e748d0700ec8b543b73
W20260811 09:12:37.722956 27796 leader_election.cc:341] T c4a7d334176e4e748d0700ec8b543b73 P 153e73b493bf47509fd161e335705439 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 82c897eb04564e7d8870a909b3082d02 (127.28.154.131:38565): Not found: Tablet not found: c4a7d334176e4e748d0700ec8b543b73
I20260811 09:12:37.723106 27796 leader_election.cc:302] T c4a7d334176e4e748d0700ec8b543b73 P 153e73b493bf47509fd161e335705439 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 153e73b493bf47509fd161e335705439; no voters: 82c897eb04564e7d8870a909b3082d02, 84334312b3254576991f3f49ecc134b1
I20260811 09:12:37.723351 28517 raft_consensus.cc:2726] T c4a7d334176e4e748d0700ec8b543b73 P 153e73b493bf47509fd161e335705439 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 09:12:38.183231 28518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:38.182304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:40403 (local address 127.28.154.131:51561)
0811 09:12:38.182925 (+ 621us) negotiation.cc:107] Waiting for socket to connect
0811 09:12:38.183018 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:40403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":510,"thread_start_us":276,"threads_started":1}
I20260811 09:12:38.242834 28521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:38.241970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:40403 (local address 127.28.154.129:45561)
0811 09:12:38.242557 (+ 587us) negotiation.cc:107] Waiting for socket to connect
0811 09:12:38.242674 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:40403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":246,"threads_started":1}
I20260811 09:12:38.722998 28522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:38.722157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:40403 (local address 127.28.154.130:38873)
0811 09:12:38.722682 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0811 09:12:38.722775 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:40403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":252,"threads_started":1}
I20260811 09:12:39.185149 28523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:39.184280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:40403 (local address 127.28.154.131:34819)
0811 09:12:39.184840 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0811 09:12:39.184932 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:40403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":336,"threads_started":1}
I20260811 09:12:39.244834 28525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:39.243925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:40403 (local address 127.28.154.129:44895)
0811 09:12:39.244515 (+ 590us) negotiation.cc:107] Waiting for socket to connect
0811 09:12:39.244609 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:40403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":263,"threads_started":1}
I20260811 09:12:39.724865 28526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:39.724056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:40403 (local address 127.28.154.130:49977)
0811 09:12:39.724598 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0811 09:12:39.724698 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:40403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":293,"threads_started":1}
I20260811 09:12:40.187014 28527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:40.186216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:40403 (local address 127.28.154.131:55901)
0811 09:12:40.186754 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0811 09:12:40.186853 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:40403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":261,"threads_started":1}
I20260811 09:12:40.246959 28528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:40.246094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:40403 (local address 127.28.154.129:51665)
0811 09:12:40.246654 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0811 09:12:40.246768 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:40403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":347,"threads_started":1}
I20260811 09:12:40.726708 28529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:40.725871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:40403 (local address 127.28.154.130:52961)
0811 09:12:40.726401 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0811 09:12:40.726508 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:40403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":270,"threads_started":1}
I20260811 09:12:41.188786 28530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:41.187969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:40403 (local address 127.28.154.131:40519)
0811 09:12:41.188486 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0811 09:12:41.188583 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:40403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":255,"threads_started":1}
I20260811 09:12:41.248950 28533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:41.248112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:40403 (local address 127.28.154.129:57777)
0811 09:12:41.248655 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0811 09:12:41.248753 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:40403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":427,"thread_start_us":305,"threads_started":1}
I20260811 09:12:41.728611 28535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:41.727768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:40403 (local address 127.28.154.130:55365)
0811 09:12:41.728299 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0811 09:12:41.728442 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:40403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":273,"threads_started":1}
I20260811 09:12:42.190513 28536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:42.189755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:40403 (local address 127.28.154.131:55665)
0811 09:12:42.190255 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0811 09:12:42.190347 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:40403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":238,"threads_started":1}
I20260811 09:12:42.250763 28537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:42.249918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:40403 (local address 127.28.154.129:37659)
0811 09:12:42.250444 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0811 09:12:42.250586 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:40403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":264,"threads_started":1}
I20260811 09:12:42.730294 28541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:42.729574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:40403 (local address 127.28.154.130:60711)
0811 09:12:42.730027 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0811 09:12:42.730117 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:40403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":262,"threads_started":1}
I20260811 09:12:43.192312 28543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:43.191512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:40403 (local address 127.28.154.131:33469)
0811 09:12:43.192034 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0811 09:12:43.192127 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:40403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":279,"threads_started":1}
I20260811 09:12:43.252703 28545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:43.251863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:40403 (local address 127.28.154.129:59557)
0811 09:12:43.252392 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0811 09:12:43.252518 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:40403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":247,"threads_started":1}
I20260811 09:12:43.732000 28546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:43.731230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:40403 (local address 127.28.154.130:57415)
0811 09:12:43.731729 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0811 09:12:43.731828 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:40403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":236,"threads_started":1}
I20260811 09:12:44.194258 28547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:44.193384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:40403 (local address 127.28.154.131:55071)
0811 09:12:44.193938 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0811 09:12:44.194063 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:40403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":266,"threads_started":1}
I20260811 09:12:44.254446 28548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:44.253681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:40403 (local address 127.28.154.129:53657)
0811 09:12:44.254161 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0811 09:12:44.254250 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:40403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":232,"threads_started":1}
I20260811 09:12:44.527462 28550 raft_consensus.cc:482] T c4a7d334176e4e748d0700ec8b543b73 P 153e73b493bf47509fd161e335705439 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:12:44.527681 28550 raft_consensus.cc:504] T c4a7d334176e4e748d0700ec8b543b73 P 153e73b493bf47509fd161e335705439 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82c897eb04564e7d8870a909b3082d02" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 38565 } } peers { permanent_uuid: "153e73b493bf47509fd161e335705439" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 41979 } } peers { permanent_uuid: "84334312b3254576991f3f49ecc134b1" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 37427 } }
I20260811 09:12:44.528367 28550 leader_election.cc:288] T c4a7d334176e4e748d0700ec8b543b73 P 153e73b493bf47509fd161e335705439 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 82c897eb04564e7d8870a909b3082d02 (127.28.154.131:38565), 84334312b3254576991f3f49ecc134b1 (127.28.154.130:37427)
W20260811 09:12:44.529357 27794 leader_election.cc:341] T c4a7d334176e4e748d0700ec8b543b73 P 153e73b493bf47509fd161e335705439 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 84334312b3254576991f3f49ecc134b1 (127.28.154.130:37427): Not found: Tablet not found: c4a7d334176e4e748d0700ec8b543b73
W20260811 09:12:44.529568 27796 leader_election.cc:341] T c4a7d334176e4e748d0700ec8b543b73 P 153e73b493bf47509fd161e335705439 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 82c897eb04564e7d8870a909b3082d02 (127.28.154.131:38565): Not found: Tablet not found: c4a7d334176e4e748d0700ec8b543b73
I20260811 09:12:44.529691 27796 leader_election.cc:302] T c4a7d334176e4e748d0700ec8b543b73 P 153e73b493bf47509fd161e335705439 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 153e73b493bf47509fd161e335705439; no voters: 82c897eb04564e7d8870a909b3082d02, 84334312b3254576991f3f49ecc134b1
I20260811 09:12:44.530087 28550 raft_consensus.cc:2726] T c4a7d334176e4e748d0700ec8b543b73 P 153e73b493bf47509fd161e335705439 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 09:12:44.733662 28551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:44.732890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:40403 (local address 127.28.154.130:58401)
0811 09:12:44.733346 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0811 09:12:44.733449 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:40403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":238,"threads_started":1}
I20260811 09:12:45.196084 28552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:45.195293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:40403 (local address 127.28.154.131:43009)
0811 09:12:45.195793 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0811 09:12:45.195884 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:40403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":257,"threads_started":1}
I20260811 09:12:45.256503 28555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:45.255735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:40403 (local address 127.28.154.129:46097)
0811 09:12:45.256201 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0811 09:12:45.256285 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:40403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":229,"threads_started":1}
I20260811 09:12:45.735499 28556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:45.734770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:40403 (local address 127.28.154.130:35185)
0811 09:12:45.735228 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0811 09:12:45.735338 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:40403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":221,"threads_started":1}
I20260811 09:12:46.176789 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 27685
I20260811 09:12:46.195271 28561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:46.178048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41084 (local address 127.28.154.129:41979)
0811 09:12:46.178575 (+ 527us) server_negotiation.cc:206] Beginning negotiation
0811 09:12:46.178584 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 09:12:46.178687 (+ 103us) server_negotiation.cc:407] Connection header received
0811 09:12:46.178834 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:12:46.178841 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:12:46.179077 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:12:46.179212 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:12:46.180711 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:46.181727 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:46.182956 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:46.183552 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:46.183650 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:12:46.189414 (+ 5764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:12:46.189431 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:12:46.189448 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:12:46.189489 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:12:46.192298 (+ 2809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:12:46.193079 (+ 781us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:12:46.193088 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:12:46.193090 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:12:46.193233 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:12:46.193604 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:12:46.193608 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:12:46.193610 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:12:46.194004 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:12:46.194334 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0811 09:12:46.194690 (+ 356us) server_negotiation.cc:299] Negotiation successful
0811 09:12:46.194968 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":204,"threads_started":1}
I20260811 09:12:46.196647 27986 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:12:46.198282 28564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:46.197297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:40403 (local address 127.28.154.131:38637)
0811 09:12:46.197897 (+ 600us) negotiation.cc:107] Waiting for socket to connect
0811 09:12:46.198071 (+ 174us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:40403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":287,"threads_started":1}
I20260811 09:12:46.293244 28567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:46.285699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:40403 (local address 127.28.154.129:39011)
0811 09:12:46.292798 (+ 7099us) negotiation.cc:107] Waiting for socket to connect
0811 09:12:46.292954 (+ 156us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.190:40403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":6969,"thread_start_us":6810,"threads_started":1}
I20260811 09:12:46.293628 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 27685
I20260811 09:12:46.318215 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 28000
I20260811 09:12:46.331943 28569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:46.319017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41842 (local address 127.28.154.130:37427)
0811 09:12:46.319455 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0811 09:12:46.319460 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:12:46.319478 (+ 18us) server_negotiation.cc:407] Connection header received
0811 09:12:46.319536 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:12:46.319540 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:12:46.319603 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:12:46.319700 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:12:46.321311 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:46.322095 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:46.323358 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:46.323811 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:46.323943 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:12:46.326771 (+ 2828us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:12:46.326776 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:12:46.326780 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:12:46.326815 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:12:46.329528 (+ 2713us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:12:46.330154 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:12:46.330158 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:12:46.330161 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:12:46.330290 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:12:46.330577 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:12:46.330580 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:12:46.330582 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:12:46.330841 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:12:46.331042 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0811 09:12:46.331429 (+ 387us) server_negotiation.cc:299] Negotiation successful
0811 09:12:46.331610 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":235,"threads_started":1}
I20260811 09:12:46.332767 28222 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:12:46.416361 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 28000
I20260811 09:12:46.442413 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 28232
I20260811 09:12:46.456276 28572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:46.443225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34146 (local address 127.28.154.131:38565)
0811 09:12:46.443762 (+ 537us) server_negotiation.cc:206] Beginning negotiation
0811 09:12:46.443767 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:12:46.443784 (+ 17us) server_negotiation.cc:407] Connection header received
0811 09:12:46.443837 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:12:46.443841 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:12:46.443913 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:12:46.444012 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:12:46.445448 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:46.446241 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:46.447485 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:46.447911 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:46.448022 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:12:46.450651 (+ 2629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:12:46.450656 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:12:46.450660 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:12:46.450693 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:12:46.453581 (+ 2888us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:12:46.454303 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:12:46.454308 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:12:46.454312 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:12:46.454472 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:12:46.454855 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:12:46.454859 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:12:46.454861 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:12:46.455124 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:12:46.455395 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0811 09:12:46.455677 (+ 282us) server_negotiation.cc:299] Negotiation successful
0811 09:12:46.455891 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":415,"thread_start_us":273,"threads_started":1}
I20260811 09:12:46.457020 28449 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:12:46.540266 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 28232
2026-08-11T09:12:46Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (65230 ms)
[----------] 30 tests from SecurityITest (316246 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439250192320-29290-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:12:46 dist-test-slave-xt4k krb5kdc[28580](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:12:46 dist-test-slave-xt4k krb5kdc[28580](info): set up 2 sockets
Aug 11 09:12:46 dist-test-slave-xt4k krb5kdc[28580](info): commencing operation
krb5kdc: starting...
W20260811 09:12:47.846621 29290 mini_kdc.cc:121] Time spent starting KDC: real 1.256s user 0.004s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439250192320-29290-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439250192320-29290-0/krb5kdc/test-user.keytab.
Aug 11 09:12:48 dist-test-slave-xt4k krb5kdc[28580](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439568, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:12:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:12:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.154.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.154.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.154.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab.
Entry for principal kudu/127.28.154.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab.
I20260811 09:12:48.155949 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439250192320-29290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439250192320-29290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439250192320-29290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.154.190:38689
--webserver_interface=127.28.154.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab
--principal=kudu/127.28.154.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:37815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.154.190:38689
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:12:48.502063 28602 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:12:48.502388 28602 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:12:48.502537 28602 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:12:48.510440 28602 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:12:48.510568 28602 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:12:48.510656 28602 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:12:48.510735 28602 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:12:48.510797 28602 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:12:48.521575 28602 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:37815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439250192320-29290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439250192320-29290-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.154.190:38689
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab
--principal=kudu/127.28.154.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.154.190:38689
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.154.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439250192320-29290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:12:48.523005 28602 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:12:48.524380 28602 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:12:48.536489 28615 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:12:48.536551 28624 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:12:48.536573 28620 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:12:48.537333 28602 server_base.cc:1034] running on GCE node
I20260811 09:12:48.538132 28602 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:12:48.539615 28602 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:12:48.540827 28602 hybrid_clock.cc:648] HybridClock initialized: now 1786439568540778 us; error 30 us; skew 500 ppm
I20260811 09:12:48.541194 28602 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:12:48 dist-test-slave-xt4k krb5kdc[28580](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439568, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:12:48.544445 28602 init.cc:377] Logged in from keytab as kudu/127.28.154.190@KRBTEST.COM (short username kudu)
I20260811 09:12:48.545995 28602 webserver.cc:460] Webserver started at http://127.28.154.190:40689/ using document root <none> and password file <none>
I20260811 09:12:48.546672 28602 fs_manager.cc:359] Metadata directory not provided
I20260811 09:12:48.546793 28602 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:12:48.547091 28602 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:12:48.550282 28602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439250192320-29290-0/minicluster-data/master-0/data/instance:
uuid: "0cf5553defee462ab9c66631e7042a74"
format_stamp: "Formatted at 2026-08-11 09:12:48 on dist-test-slave-xt4k"
server_key: "2ad6de781e407ea6986b8fac2bdfd688"
server_key_iv: "ceeff915ec9a08226f5e92be1a003b0a"
server_key_version: "encryptionkey@0"
I20260811 09:12:48.551057 28602 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439250192320-29290-0/minicluster-data/master-0/wal/instance:
uuid: "0cf5553defee462ab9c66631e7042a74"
format_stamp: "Formatted at 2026-08-11 09:12:48 on dist-test-slave-xt4k"
server_key: "2ad6de781e407ea6986b8fac2bdfd688"
server_key_iv: "ceeff915ec9a08226f5e92be1a003b0a"
server_key_version: "encryptionkey@0"
I20260811 09:12:48.556443 28602 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260811 09:12:48.559948 28637 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:12:48.561350 28602 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 09:12:48.561518 28602 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439250192320-29290-0/minicluster-data/master-0/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439250192320-29290-0/minicluster-data/master-0/wal
uuid: "0cf5553defee462ab9c66631e7042a74"
format_stamp: "Formatted at 2026-08-11 09:12:48 on dist-test-slave-xt4k"
server_key: "2ad6de781e407ea6986b8fac2bdfd688"
server_key_iv: "ceeff915ec9a08226f5e92be1a003b0a"
server_key_version: "encryptionkey@0"
I20260811 09:12:48.561722 28602 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439250192320-29290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439250192320-29290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439250192320-29290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:12:48.580616 28602 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:12:48.584765 28602 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:12:48.585036 28602 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:12:48.603840 28602 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.190:38689
I20260811 09:12:48.603860 28721 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.190:38689 every 8 connection(s)
I20260811 09:12:48.605263 28602 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
I20260811 09:12:48.607363 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 28602
I20260811 09:12:48.607604 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439250192320-29290-0/minicluster-data/master-0/wal/instance
I20260811 09:12:48.608474 29290 external_mini_cluster.cc:1428] Setting key 00fcf452346a548cb241a58601f5fca2
I20260811 09:12:48.609915 28723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 09:12:48 dist-test-slave-xt4k krb5kdc[28580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439568, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.154.190@KRBTEST.COM
I20260811 09:12:48.625000 28723 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0cf5553defee462ab9c66631e7042a74: Bootstrap starting.
I20260811 09:12:48.628422 28723 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0cf5553defee462ab9c66631e7042a74: Neither blocks nor log segments found. Creating new log.
I20260811 09:12:48.629647 28723 log.cc:824] T 00000000000000000000000000000000 P 0cf5553defee462ab9c66631e7042a74: Log is configured to *not* fsync() on all Append() calls
I20260811 09:12:48.632969 28723 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0cf5553defee462ab9c66631e7042a74: No bootstrap required, opened a new log
I20260811 09:12:48.633396 28727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:48.610673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60788 (local address 127.28.154.190:38689)
0811 09:12:48.611683 (+ 1010us) server_negotiation.cc:206] Beginning negotiation
0811 09:12:48.611698 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 09:12:48.611744 (+ 46us) server_negotiation.cc:407] Connection header received
0811 09:12:48.613615 (+ 1871us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:12:48.613639 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:12:48.614056 (+ 417us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:12:48.614416 (+ 360us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:12:48.616378 (+ 1962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:48.617831 (+ 1453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:48.619761 (+ 1930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:48.620567 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:48.620721 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:12:48.625832 (+ 5111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:12:48.625841 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:12:48.625858 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:12:48.625907 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:12:48.628966 (+ 3059us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:12:48.629884 (+ 918us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:12:48.629891 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:12:48.629898 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:12:48.630073 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:12:48.630512 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:12:48.630517 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:12:48.630520 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:12:48.631211 (+ 691us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:12:48.631429 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0811 09:12:48.631965 (+ 536us) server_negotiation.cc:299] Negotiation successful
0811 09:12:48.632319 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":590,"thread_start_us":362,"threads_started":1}
I20260811 09:12:48.641281 28723 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0cf5553defee462ab9c66631e7042a74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cf5553defee462ab9c66631e7042a74" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 38689 } }
I20260811 09:12:48.641672 28723 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0cf5553defee462ab9c66631e7042a74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:12:48.641768 28723 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0cf5553defee462ab9c66631e7042a74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cf5553defee462ab9c66631e7042a74, State: Initialized, Role: FOLLOWER
I20260811 09:12:48.642318 28723 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0cf5553defee462ab9c66631e7042a74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cf5553defee462ab9c66631e7042a74" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 38689 } }
I20260811 09:12:48.642563 28723 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0cf5553defee462ab9c66631e7042a74 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:12:48.642696 28723 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0cf5553defee462ab9c66631e7042a74 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:12:48.642850 28723 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0cf5553defee462ab9c66631e7042a74 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:12:48.645332 28723 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0cf5553defee462ab9c66631e7042a74 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cf5553defee462ab9c66631e7042a74" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 38689 } }
I20260811 09:12:48.645819 28723 leader_election.cc:302] T 00000000000000000000000000000000 P 0cf5553defee462ab9c66631e7042a74 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0cf5553defee462ab9c66631e7042a74; no voters:
I20260811 09:12:48.646413 28723 leader_election.cc:288] T 00000000000000000000000000000000 P 0cf5553defee462ab9c66631e7042a74 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:12:48.646533 28730 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0cf5553defee462ab9c66631e7042a74 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:12:48.647207 28730 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0cf5553defee462ab9c66631e7042a74 [term 1 LEADER]: Becoming Leader. State: Replica: 0cf5553defee462ab9c66631e7042a74, State: Running, Role: LEADER
I20260811 09:12:48.647758 28730 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0cf5553defee462ab9c66631e7042a74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cf5553defee462ab9c66631e7042a74" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 38689 } }
I20260811 09:12:48.648442 28723 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0cf5553defee462ab9c66631e7042a74 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:12:48.650460 28734 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0cf5553defee462ab9c66631e7042a74 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0cf5553defee462ab9c66631e7042a74. Latest consensus state: current_term: 1 leader_uuid: "0cf5553defee462ab9c66631e7042a74" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cf5553defee462ab9c66631e7042a74" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 38689 } } }
I20260811 09:12:48.650882 28734 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0cf5553defee462ab9c66631e7042a74 [sys.catalog]: This master's current role is: LEADER
I20260811 09:12:48.651067 28733 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0cf5553defee462ab9c66631e7042a74 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0cf5553defee462ab9c66631e7042a74" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cf5553defee462ab9c66631e7042a74" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 38689 } } }
I20260811 09:12:48.653254 28733 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0cf5553defee462ab9c66631e7042a74 [sys.catalog]: This master's current role is: LEADER
I20260811 09:12:48.653882 28742 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:12:48.658571 28742 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:12:48.668359 28742 catalog_manager.cc:1347] Generated new cluster ID: 48dacfbe1df74527a2be489b832dc023
I20260811 09:12:48.668490 28742 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:12:48.684743 28742 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:12:48.686151 28742 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:12:48.701416 28742 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0cf5553defee462ab9c66631e7042a74: Generated new TSK 0
I20260811 09:12:48.702576 28742 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.28.154.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.154.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.154.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.129.keytab.
Entry for principal kudu/127.28.154.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.129.keytab.
I20260811 09:12:48.779701 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439250192320-29290-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439250192320-29290-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439250192320-29290-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439250192320-29290-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.154.129:0
--local_ip_for_outbound_sockets=127.28.154.129
--webserver_interface=127.28.154.129
--webserver_port=0
--tserver_master_addrs=127.28.154.190:38689
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.129.keytab
--principal=kudu/127.28.154.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:37815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:12:49.177819 28770 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:12:49.178114 28770 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:12:49.178210 28770 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:12:49.185698 28770 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:12:49.185817 28770 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:12:49.185941 28770 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.154.129
I20260811 09:12:49.196081 28770 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:37815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439250192320-29290-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439250192320-29290-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.129.keytab
--principal=kudu/127.28.154.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.154.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439250192320-29290-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.154.129
--webserver_port=0
--tserver_master_addrs=127.28.154.190:38689
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.154.129
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439250192320-29290-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:12:49.197454 28770 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:12:49.198743 28770 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:12:49.212368 28780 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:12:49.212285 28781 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:12:49.212322 28783 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:12:49.213557 28770 server_base.cc:1034] running on GCE node
I20260811 09:12:49.214192 28770 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:12:49.215248 28770 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:12:49.216460 28770 hybrid_clock.cc:648] HybridClock initialized: now 1786439569216403 us; error 40 us; skew 500 ppm
I20260811 09:12:49.216805 28770 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:12:49 dist-test-slave-xt4k krb5kdc[28580](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439569, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:12:49.220127 28770 init.cc:377] Logged in from keytab as kudu/127.28.154.129@KRBTEST.COM (short username kudu)
I20260811 09:12:49.221782 28770 webserver.cc:460] Webserver started at http://127.28.154.129:33549/ using document root <none> and password file <none>
I20260811 09:12:49.222505 28770 fs_manager.cc:359] Metadata directory not provided
I20260811 09:12:49.222623 28770 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:12:49.222913 28770 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:12:49.226014 28770 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439250192320-29290-0/minicluster-data/ts-0/data/instance:
uuid: "9557d17aae1b4c8c962a0fbb25048155"
format_stamp: "Formatted at 2026-08-11 09:12:49 on dist-test-slave-xt4k"
server_key: "09a3bdfbea0fe4b400de98be15fb09ab"
server_key_iv: "3488b269452e5f9e576f96998322e6ec"
server_key_version: "encryptionkey@0"
I20260811 09:12:49.226778 28770 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439250192320-29290-0/minicluster-data/ts-0/wal/instance:
uuid: "9557d17aae1b4c8c962a0fbb25048155"
format_stamp: "Formatted at 2026-08-11 09:12:49 on dist-test-slave-xt4k"
server_key: "09a3bdfbea0fe4b400de98be15fb09ab"
server_key_iv: "3488b269452e5f9e576f96998322e6ec"
server_key_version: "encryptionkey@0"
I20260811 09:12:49.232297 28770 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.003s
I20260811 09:12:49.236109 28791 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:12:49.237618 28770 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 09:12:49.237815 28770 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439250192320-29290-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439250192320-29290-0/minicluster-data/ts-0/wal
uuid: "9557d17aae1b4c8c962a0fbb25048155"
format_stamp: "Formatted at 2026-08-11 09:12:49 on dist-test-slave-xt4k"
server_key: "09a3bdfbea0fe4b400de98be15fb09ab"
server_key_iv: "3488b269452e5f9e576f96998322e6ec"
server_key_version: "encryptionkey@0"
I20260811 09:12:49.238022 28770 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439250192320-29290-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439250192320-29290-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439250192320-29290-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:12:49.260238 28770 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:12:49.264868 28770 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:12:49.265167 28770 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:12:49.266224 28770 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:12:49.267678 28770 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:12:49.267755 28770 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:12:49.267853 28770 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:12:49.267895 28770 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:12:49.304996 28770 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.129:35101
I20260811 09:12:49.305058 29009 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.129:35101 every 8 connection(s)
I20260811 09:12:49.306380 28770 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439250192320-29290-0/minicluster-data/ts-0/data/info.pb
I20260811 09:12:49.316767 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 28770
I20260811 09:12:49.317005 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439250192320-29290-0/minicluster-data/ts-0/wal/instance
I20260811 09:12:49.317687 29290 external_mini_cluster.cc:1428] Setting key 238997d1c025ce9e2af4b2943fd12381
Aug 11 09:12:49 dist-test-slave-xt4k krb5kdc[28580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439569, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.129@KRBTEST.COM for kudu/127.28.154.190@KRBTEST.COM
I20260811 09:12:49.327243 29016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:49.309152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.129:45999 (local address 127.28.154.190:38689)
0811 09:12:49.309581 (+ 429us) server_negotiation.cc:206] Beginning negotiation
0811 09:12:49.309585 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:12:49.310645 (+ 1060us) server_negotiation.cc:407] Connection header received
0811 09:12:49.312599 (+ 1954us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:12:49.312605 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:12:49.312690 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:12:49.312790 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:12:49.314392 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:49.315123 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:49.316730 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:49.317203 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:49.317316 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:12:49.321258 (+ 3942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:12:49.321263 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:12:49.321267 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:12:49.321302 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:12:49.323863 (+ 2561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:12:49.324803 (+ 940us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:12:49.324808 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:12:49.324810 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:12:49.324949 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:12:49.325394 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:12:49.325398 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:12:49.325400 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:12:49.325772 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:12:49.325912 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0811 09:12:49.326777 (+ 865us) server_negotiation.cc:299] Negotiation successful
0811 09:12:49.326972 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":214,"threads_started":1}
I20260811 09:12:49.328622 29017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:49.309667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:38689 (local address 127.28.154.129:45999)
0811 09:12:49.310467 (+ 800us) negotiation.cc:107] Waiting for socket to connect
0811 09:12:49.310507 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 09:12:49.312240 (+ 1733us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:12:49.313053 (+ 813us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:12:49.313061 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:12:49.313606 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:12:49.314207 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:12:49.314225 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:12:49.315282 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:12:49.315286 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:12:49.316588 (+ 1302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:12:49.316602 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:12:49.317454 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:12:49.317462 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:12:49.317686 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:12:49.318424 (+ 738us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:12:49.318508 (+ 84us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:12:49.321009 (+ 2501us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:12:49.324011 (+ 3002us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:12:49.324020 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:12:49.324038 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:12:49.324640 (+ 602us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:12:49.325052 (+ 412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:12:49.325057 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:12:49.325060 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:12:49.325299 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:12:49.325906 (+ 607us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:12:49.325913 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:12:49.326360 (+ 447us) client_negotiation.cc:769] Sending connection context
0811 09:12:49.326663 (+ 303us) client_negotiation.cc:240] Negotiation successful
0811 09:12:49.327212 (+ 549us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":509,"thread_start_us":291,"threads_started":1}
I20260811 09:12:49.330281 29010 heartbeater.cc:347] Connected to a master server at 127.28.154.190:38689
I20260811 09:12:49.330618 29010 heartbeater.cc:464] Registering TS with master...
I20260811 09:12:49.331507 29010 heartbeater.cc:511] Master 127.28.154.190:38689 requested a full tablet report, sending...
I20260811 09:12:49.334244 28661 ts_manager.cc:194] Registered new tserver with Master: 9557d17aae1b4c8c962a0fbb25048155 (127.28.154.129:35101)
I20260811 09:12:49.336223 28661 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.154.129@KRBTEST.COM'} at 127.28.154.129:45999
WARNING: no policy specified for kudu/127.28.154.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.154.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.154.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.130.keytab.
Entry for principal kudu/127.28.154.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.130.keytab.
I20260811 09:12:49.384114 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439250192320-29290-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439250192320-29290-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439250192320-29290-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439250192320-29290-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.154.130:0
--local_ip_for_outbound_sockets=127.28.154.130
--webserver_interface=127.28.154.130
--webserver_port=0
--tserver_master_addrs=127.28.154.190:38689
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.130.keytab
--principal=kudu/127.28.154.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:37815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:12:49.724216 29021 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:12:49.724537 29021 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:12:49.724650 29021 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:12:49.732064 29021 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:12:49.732184 29021 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:12:49.732309 29021 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.154.130
I20260811 09:12:49.742861 29021 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:37815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439250192320-29290-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439250192320-29290-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.130.keytab
--principal=kudu/127.28.154.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.154.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439250192320-29290-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.154.130
--webserver_port=0
--tserver_master_addrs=127.28.154.190:38689
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.154.130
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439250192320-29290-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:12:49.744239 29021 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:12:49.745942 29021 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:12:49.758873 29031 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:12:49.758914 29037 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:12:49.758941 29033 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:12:49.759773 29021 server_base.cc:1034] running on GCE node
I20260811 09:12:49.760464 29021 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:12:49.761380 29021 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:12:49.762584 29021 hybrid_clock.cc:648] HybridClock initialized: now 1786439569762480 us; error 97 us; skew 500 ppm
I20260811 09:12:49.762883 29021 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:12:49 dist-test-slave-xt4k krb5kdc[28580](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439569, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:12:49.766062 29021 init.cc:377] Logged in from keytab as kudu/127.28.154.130@KRBTEST.COM (short username kudu)
I20260811 09:12:49.767457 29021 webserver.cc:460] Webserver started at http://127.28.154.130:44141/ using document root <none> and password file <none>
I20260811 09:12:49.768124 29021 fs_manager.cc:359] Metadata directory not provided
I20260811 09:12:49.768232 29021 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:12:49.768556 29021 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:12:49.771345 29021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439250192320-29290-0/minicluster-data/ts-1/data/instance:
uuid: "4cabf1c01fa24edf81d1f561252ba26e"
format_stamp: "Formatted at 2026-08-11 09:12:49 on dist-test-slave-xt4k"
server_key: "5adf9a84e831b80e0fe75ac907d9561a"
server_key_iv: "b1be2236fff4b49f41b72ce3f58b4370"
server_key_version: "encryptionkey@0"
I20260811 09:12:49.772068 29021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439250192320-29290-0/minicluster-data/ts-1/wal/instance:
uuid: "4cabf1c01fa24edf81d1f561252ba26e"
format_stamp: "Formatted at 2026-08-11 09:12:49 on dist-test-slave-xt4k"
server_key: "5adf9a84e831b80e0fe75ac907d9561a"
server_key_iv: "b1be2236fff4b49f41b72ce3f58b4370"
server_key_version: "encryptionkey@0"
I20260811 09:12:49.777122 29021 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260811 09:12:49.780400 29049 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:12:49.781648 29021 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 09:12:49.781788 29021 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439250192320-29290-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439250192320-29290-0/minicluster-data/ts-1/wal
uuid: "4cabf1c01fa24edf81d1f561252ba26e"
format_stamp: "Formatted at 2026-08-11 09:12:49 on dist-test-slave-xt4k"
server_key: "5adf9a84e831b80e0fe75ac907d9561a"
server_key_iv: "b1be2236fff4b49f41b72ce3f58b4370"
server_key_version: "encryptionkey@0"
I20260811 09:12:49.781965 29021 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439250192320-29290-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439250192320-29290-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439250192320-29290-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:12:49.796723 29021 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:12:49.802069 29021 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:12:49.802412 29021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:12:49.803742 29021 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:12:49.807159 29021 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:12:49.807281 29021 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260811 09:12:49.807418 29021 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:12:49.807512 29021 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:12:49.848611 29021 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.130:36817
I20260811 09:12:49.848680 29245 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.130:36817 every 8 connection(s)
I20260811 09:12:49.849939 29021 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439250192320-29290-0/minicluster-data/ts-1/data/info.pb
I20260811 09:12:49.859802 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 29021
I20260811 09:12:49.859988 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439250192320-29290-0/minicluster-data/ts-1/wal/instance
I20260811 09:12:49.860473 29290 external_mini_cluster.cc:1428] Setting key 70f5b0aec21b922425cd70e32df37c30
Aug 11 09:12:49 dist-test-slave-xt4k krb5kdc[28580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439569, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.130@KRBTEST.COM for kudu/127.28.154.190@KRBTEST.COM
I20260811 09:12:49.869490 29260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:49.852328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.130:44169 (local address 127.28.154.190:38689)
0811 09:12:49.852801 (+ 473us) server_negotiation.cc:206] Beginning negotiation
0811 09:12:49.852806 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:12:49.853629 (+ 823us) server_negotiation.cc:407] Connection header received
0811 09:12:49.855481 (+ 1852us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:12:49.855487 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:12:49.855566 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:12:49.855671 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:12:49.857309 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:49.858042 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:49.859541 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:49.860039 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:49.860159 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:12:49.863990 (+ 3831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:12:49.863996 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:12:49.863999 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:12:49.864031 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:12:49.866500 (+ 2469us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:12:49.867321 (+ 821us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:12:49.867325 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:12:49.867327 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:12:49.867455 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:12:49.867898 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:12:49.867902 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:12:49.867904 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:12:49.868152 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:12:49.868256 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 09:12:49.869062 (+ 806us) server_negotiation.cc:299] Negotiation successful
0811 09:12:49.869222 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":347,"thread_start_us":228,"threads_started":1}
I20260811 09:12:49.870775 29261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:49.852767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:38689 (local address 127.28.154.130:44169)
0811 09:12:49.853468 (+ 701us) negotiation.cc:107] Waiting for socket to connect
0811 09:12:49.853513 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 09:12:49.855218 (+ 1705us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:12:49.855922 (+ 704us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:12:49.855929 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:12:49.856536 (+ 607us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:12:49.857131 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:12:49.857152 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:12:49.858196 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:12:49.858200 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:12:49.859403 (+ 1203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:12:49.859415 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:12:49.860258 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:12:49.860265 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:12:49.860531 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:12:49.861235 (+ 704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:12:49.861266 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:12:49.863756 (+ 2490us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:12:49.866638 (+ 2882us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:12:49.866646 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:12:49.866662 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:12:49.867201 (+ 539us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:12:49.867548 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:12:49.867553 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:12:49.867556 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:12:49.867782 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:12:49.868260 (+ 478us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:12:49.868267 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:12:49.868692 (+ 425us) client_negotiation.cc:769] Sending connection context
0811 09:12:49.868964 (+ 272us) client_negotiation.cc:240] Negotiation successful
0811 09:12:49.869473 (+ 509us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":272,"threads_started":1}
I20260811 09:12:49.872193 29247 heartbeater.cc:347] Connected to a master server at 127.28.154.190:38689
I20260811 09:12:49.872510 29247 heartbeater.cc:464] Registering TS with master...
I20260811 09:12:49.873245 29247 heartbeater.cc:511] Master 127.28.154.190:38689 requested a full tablet report, sending...
I20260811 09:12:49.874642 28661 ts_manager.cc:194] Registered new tserver with Master: 4cabf1c01fa24edf81d1f561252ba26e (127.28.154.130:36817)
I20260811 09:12:49.875447 28661 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.154.130@KRBTEST.COM'} at 127.28.154.130:44169
WARNING: no policy specified for kudu/127.28.154.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.154.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.154.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.131.keytab.
Entry for principal kudu/127.28.154.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.131.keytab.
I20260811 09:12:49.923501 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439250192320-29290-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439250192320-29290-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439250192320-29290-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439250192320-29290-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.154.131:0
--local_ip_for_outbound_sockets=127.28.154.131
--webserver_interface=127.28.154.131
--webserver_port=0
--tserver_master_addrs=127.28.154.190:38689
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.131.keytab
--principal=kudu/127.28.154.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:37815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:12:50.263654 29266 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:12:50.263980 29266 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:12:50.264111 29266 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:12:50.271548 29266 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:12:50.271687 29266 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:12:50.271842 29266 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.154.131
I20260811 09:12:50.281886 29266 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:37815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439250192320-29290-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439250192320-29290-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.131.keytab
--principal=kudu/127.28.154.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.154.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439250192320-29290-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.154.131
--webserver_port=0
--tserver_master_addrs=127.28.154.190:38689
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.154.131
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439250192320-29290-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:12:50.283227 29266 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:12:50.284600 29266 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:12:50.298147 29278 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:12:50.298149 29273 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:12:50.298422 29271 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:12:50.298811 29266 server_base.cc:1034] running on GCE node
I20260811 09:12:50.299484 29266 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:12:50.300462 29266 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:12:50.301657 29266 hybrid_clock.cc:648] HybridClock initialized: now 1786439570301599 us; error 55 us; skew 500 ppm
I20260811 09:12:50.301980 29266 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:12:50 dist-test-slave-xt4k krb5kdc[28580](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439570, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:12:50.305228 29266 init.cc:377] Logged in from keytab as kudu/127.28.154.131@KRBTEST.COM (short username kudu)
I20260811 09:12:50.308673 29266 webserver.cc:460] Webserver started at http://127.28.154.131:43957/ using document root <none> and password file <none>
I20260811 09:12:50.309372 29266 fs_manager.cc:359] Metadata directory not provided
I20260811 09:12:50.309499 29266 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:12:50.309815 29266 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:12:50.312875 29266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439250192320-29290-0/minicluster-data/ts-2/data/instance:
uuid: "e9b213925dca400bbb27881af693e926"
format_stamp: "Formatted at 2026-08-11 09:12:50 on dist-test-slave-xt4k"
server_key: "14decaf342e73237209d790d4d76179b"
server_key_iv: "83a45454861a380c259cc1ab1ed50953"
server_key_version: "encryptionkey@0"
I20260811 09:12:50.313648 29266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439250192320-29290-0/minicluster-data/ts-2/wal/instance:
uuid: "e9b213925dca400bbb27881af693e926"
format_stamp: "Formatted at 2026-08-11 09:12:50 on dist-test-slave-xt4k"
server_key: "14decaf342e73237209d790d4d76179b"
server_key_iv: "83a45454861a380c259cc1ab1ed50953"
server_key_version: "encryptionkey@0"
I20260811 09:12:50.319253 29266 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260811 09:12:50.323237 29289 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:12:50.324816 29266 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 09:12:50.324993 29266 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439250192320-29290-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439250192320-29290-0/minicluster-data/ts-2/wal
uuid: "e9b213925dca400bbb27881af693e926"
format_stamp: "Formatted at 2026-08-11 09:12:50 on dist-test-slave-xt4k"
server_key: "14decaf342e73237209d790d4d76179b"
server_key_iv: "83a45454861a380c259cc1ab1ed50953"
server_key_version: "encryptionkey@0"
I20260811 09:12:50.325217 29266 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439250192320-29290-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439250192320-29290-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439250192320-29290-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:12:50.339608 29010 heartbeater.cc:503] Master 127.28.154.190:38689 was elected leader, sending a full tablet report...
I20260811 09:12:50.364668 29266 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:12:50.369613 29266 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:12:50.369920 29266 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:12:50.370970 29266 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:12:50.372537 29266 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:12:50.372638 29266 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:12:50.372773 29266 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:12:50.372828 29266 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:12:50.406558 29266 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.131:35071
I20260811 09:12:50.406632 29501 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.131:35071 every 8 connection(s)
I20260811 09:12:50.407895 29266 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439250192320-29290-0/minicluster-data/ts-2/data/info.pb
I20260811 09:12:50.414189 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 29266
I20260811 09:12:50.414410 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439250192320-29290-0/minicluster-data/ts-2/wal/instance
I20260811 09:12:50.414872 29290 external_mini_cluster.cc:1428] Setting key 3ef4e0d968cd181d0ab75327675c3db1
Aug 11 09:12:50 dist-test-slave-xt4k krb5kdc[28580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439570, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.131@KRBTEST.COM for kudu/127.28.154.190@KRBTEST.COM
I20260811 09:12:50.430433 29505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:50.410666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.131:35825 (local address 127.28.154.190:38689)
0811 09:12:50.411128 (+ 462us) server_negotiation.cc:206] Beginning negotiation
0811 09:12:50.411133 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:12:50.412131 (+ 998us) server_negotiation.cc:407] Connection header received
0811 09:12:50.414045 (+ 1914us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:12:50.414052 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:12:50.414144 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:12:50.414269 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:12:50.416250 (+ 1981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:50.417324 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:50.419435 (+ 2111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:50.419894 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:50.420003 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:12:50.423660 (+ 3657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:12:50.423665 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:12:50.423670 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:12:50.423714 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:12:50.426605 (+ 2891us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:12:50.427533 (+ 928us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:12:50.427538 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:12:50.427540 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:12:50.427702 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:12:50.428229 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:12:50.428237 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:12:50.428239 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:12:50.428638 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:12:50.428770 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 09:12:50.429750 (+ 980us) server_negotiation.cc:299] Negotiation successful
0811 09:12:50.430005 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":213,"threads_started":1}
I20260811 09:12:50.431440 29507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:50.411192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:38689 (local address 127.28.154.131:35825)
0811 09:12:50.411954 (+ 762us) negotiation.cc:107] Waiting for socket to connect
0811 09:12:50.412003 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 09:12:50.413739 (+ 1736us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:12:50.414595 (+ 856us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:12:50.414605 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:12:50.415191 (+ 586us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:12:50.416032 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:12:50.416062 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:12:50.417573 (+ 1511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:12:50.417580 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:12:50.419278 (+ 1698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:12:50.419293 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:12:50.420109 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:12:50.420115 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:12:50.420332 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:12:50.421036 (+ 704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:12:50.421067 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:12:50.423395 (+ 2328us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:12:50.426846 (+ 3451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:12:50.426855 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:12:50.426871 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:12:50.427356 (+ 485us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:12:50.427834 (+ 478us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:12:50.427838 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:12:50.427841 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:12:50.428076 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:12:50.428779 (+ 703us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:12:50.428786 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:12:50.429211 (+ 425us) client_negotiation.cc:769] Sending connection context
0811 09:12:50.429530 (+ 319us) client_negotiation.cc:240] Negotiation successful
0811 09:12:50.430117 (+ 587us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":322,"threads_started":1}
I20260811 09:12:50.432811 29503 heartbeater.cc:347] Connected to a master server at 127.28.154.190:38689
I20260811 09:12:50.433156 29503 heartbeater.cc:464] Registering TS with master...
I20260811 09:12:50.433933 29503 heartbeater.cc:511] Master 127.28.154.190:38689 requested a full tablet report, sending...
I20260811 09:12:50.435611 28661 ts_manager.cc:194] Registered new tserver with Master: e9b213925dca400bbb27881af693e926 (127.28.154.131:35071)
I20260811 09:12:50.436681 28661 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.154.131@KRBTEST.COM'} at 127.28.154.131:35825
I20260811 09:12:50.444008 29290 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 09:12:50.462745 29505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:50.448836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60798 (local address 127.28.154.190:38689)
0811 09:12:50.449061 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0811 09:12:50.449067 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:12:50.449390 (+ 323us) server_negotiation.cc:407] Connection header received
0811 09:12:50.449561 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:12:50.449565 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:12:50.449652 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:12:50.449745 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:12:50.451263 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:50.452004 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:50.453413 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:50.453816 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:50.453931 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:12:50.456514 (+ 2583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:12:50.456520 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:12:50.456523 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:12:50.456565 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:12:50.459800 (+ 3235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:12:50.460643 (+ 843us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:12:50.460647 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:12:50.460650 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:12:50.460784 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:12:50.461169 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:12:50.461173 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:12:50.461175 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:12:50.461541 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:12:50.461788 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0811 09:12:50.462166 (+ 378us) server_negotiation.cc:299] Negotiation successful
0811 09:12:50.462429 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 09:12:50.467641 28661 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60798:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 09:12:50.475636 28661 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 09:12:50.516064 29526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:50.490632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57974 (local address 127.28.154.129:35101)
0811 09:12:50.505182 (+ 14550us) server_negotiation.cc:206] Beginning negotiation
0811 09:12:50.505193 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 09:12:50.505239 (+ 46us) server_negotiation.cc:407] Connection header received
0811 09:12:50.505378 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:12:50.505390 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:12:50.505749 (+ 359us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:12:50.507218 (+ 1469us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:12:50.507524 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:50.509359 (+ 1835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:50.513001 (+ 3642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:50.514468 (+ 1467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:50.514653 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:12:50.514837 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0811 09:12:50.515538 (+ 701us) server_negotiation.cc:299] Negotiation successful
0811 09:12:50.515725 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14284,"thread_start_us":15349,"threads_started":1}
I20260811 09:12:50.516474 29522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:50.489954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.129:35101 (local address 127.0.0.1:57974)
0811 09:12:50.500975 (+ 11021us) negotiation.cc:107] Waiting for socket to connect
0811 09:12:50.501011 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 09:12:50.501202 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:12:50.506390 (+ 5188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:12:50.506401 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:12:50.506449 (+ 48us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:12:50.506953 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:12:50.506967 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:12:50.510138 (+ 3171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:12:50.510145 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:12:50.512837 (+ 2692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:12:50.512849 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:12:50.515021 (+ 2172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:12:50.515027 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:12:50.515252 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:12:50.515319 (+ 67us) client_negotiation.cc:769] Sending connection context
0811 09:12:50.515476 (+ 157us) client_negotiation.cc:240] Negotiation successful
0811 09:12:50.515806 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10836,"mutex_wait_us":87,"thread_start_us":10618,"threads_started":1}
I20260811 09:12:50.519097 29518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:50.489142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.130:36817 (local address 127.0.0.1:40632)
0811 09:12:50.490031 (+ 889us) negotiation.cc:107] Waiting for socket to connect
0811 09:12:50.490083 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 09:12:50.490274 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:12:50.512134 (+ 21860us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:12:50.512140 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:12:50.512170 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:12:50.512651 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:12:50.512665 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:12:50.514326 (+ 1661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:12:50.514331 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:12:50.516476 (+ 2145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:12:50.516490 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:12:50.518312 (+ 1822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:12:50.518318 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:12:50.518497 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:12:50.518532 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 09:12:50.518609 (+ 77us) client_negotiation.cc:240] Negotiation successful
0811 09:12:50.518741 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":570,"thread_start_us":373,"threads_started":1}
I20260811 09:12:50.519748 29523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:50.490415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.131:35071 (local address 127.0.0.1:54700)
0811 09:12:50.510770 (+ 20355us) negotiation.cc:107] Waiting for socket to connect
0811 09:12:50.510804 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0811 09:12:50.510945 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:12:50.512535 (+ 1590us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:12:50.512540 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:12:50.512575 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:12:50.513068 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:12:50.513083 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:12:50.514939 (+ 1856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:12:50.514943 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:12:50.517242 (+ 2299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:12:50.517255 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:12:50.518943 (+ 1688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:12:50.518949 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:12:50.519139 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:12:50.519171 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 09:12:50.519270 (+ 99us) client_negotiation.cc:240] Negotiation successful
0811 09:12:50.519398 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20159,"mutex_wait_us":2,"thread_start_us":19956,"threads_started":1}
I20260811 09:12:50.519825 29527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:50.491015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54700 (local address 127.28.154.131:35071)
0811 09:12:50.511466 (+ 20451us) server_negotiation.cc:206] Beginning negotiation
0811 09:12:50.511473 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:12:50.511506 (+ 33us) server_negotiation.cc:407] Connection header received
0811 09:12:50.512065 (+ 559us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:12:50.512083 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:12:50.512309 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:12:50.512512 (+ 203us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:12:50.513253 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:50.514753 (+ 1500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:50.517418 (+ 2665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:50.518740 (+ 1322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:50.518943 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:12:50.519123 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0811 09:12:50.519323 (+ 200us) server_negotiation.cc:299] Negotiation successful
0811 09:12:50.519483 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18867,"thread_start_us":18700,"threads_started":1}
I20260811 09:12:50.520546 29521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:50.489219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40632 (local address 127.28.154.130:36817)
0811 09:12:50.511462 (+ 22243us) server_negotiation.cc:206] Beginning negotiation
0811 09:12:50.511472 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 09:12:50.511506 (+ 34us) server_negotiation.cc:407] Connection header received
0811 09:12:50.511613 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:12:50.511623 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:12:50.511915 (+ 292us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:12:50.512146 (+ 231us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:12:50.512878 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:50.514173 (+ 1295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:50.516670 (+ 2497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:50.518092 (+ 1422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:50.519543 (+ 1451us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:12:50.519721 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0811 09:12:50.519800 (+ 79us) server_negotiation.cc:299] Negotiation successful
0811 09:12:50.520207 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22107,"thread_start_us":21977,"threads_started":1}
I20260811 09:12:50.520911 28896 tablet_service.cc:1467] Processing CreateTablet for tablet ad7f0381ac1c425d952bced9f1f9f583 (DEFAULT_TABLE table=test-table [id=c833994cbfe7410492d2a2ea33cc7b2e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:12:50.522666 28896 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad7f0381ac1c425d952bced9f1f9f583. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:12:50.523765 29124 tablet_service.cc:1467] Processing CreateTablet for tablet ad7f0381ac1c425d952bced9f1f9f583 (DEFAULT_TABLE table=test-table [id=c833994cbfe7410492d2a2ea33cc7b2e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:12:50.523881 29375 tablet_service.cc:1467] Processing CreateTablet for tablet ad7f0381ac1c425d952bced9f1f9f583 (DEFAULT_TABLE table=test-table [id=c833994cbfe7410492d2a2ea33cc7b2e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:12:50.525472 29124 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad7f0381ac1c425d952bced9f1f9f583. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:12:50.525485 29375 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad7f0381ac1c425d952bced9f1f9f583. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:12:50.541358 29532 tablet_bootstrap.cc:492] T ad7f0381ac1c425d952bced9f1f9f583 P 9557d17aae1b4c8c962a0fbb25048155: Bootstrap starting.
I20260811 09:12:50.546589 29532 tablet_bootstrap.cc:654] T ad7f0381ac1c425d952bced9f1f9f583 P 9557d17aae1b4c8c962a0fbb25048155: Neither blocks nor log segments found. Creating new log.
I20260811 09:12:50.548328 29532 log.cc:824] T ad7f0381ac1c425d952bced9f1f9f583 P 9557d17aae1b4c8c962a0fbb25048155: Log is configured to *not* fsync() on all Append() calls
I20260811 09:12:50.549844 29529 tablet_bootstrap.cc:492] T ad7f0381ac1c425d952bced9f1f9f583 P 4cabf1c01fa24edf81d1f561252ba26e: Bootstrap starting.
I20260811 09:12:50.550941 29533 tablet_bootstrap.cc:492] T ad7f0381ac1c425d952bced9f1f9f583 P e9b213925dca400bbb27881af693e926: Bootstrap starting.
I20260811 09:12:50.554041 29529 tablet_bootstrap.cc:654] T ad7f0381ac1c425d952bced9f1f9f583 P 4cabf1c01fa24edf81d1f561252ba26e: Neither blocks nor log segments found. Creating new log.
I20260811 09:12:50.554386 29532 tablet_bootstrap.cc:492] T ad7f0381ac1c425d952bced9f1f9f583 P 9557d17aae1b4c8c962a0fbb25048155: No bootstrap required, opened a new log
I20260811 09:12:50.554738 29532 ts_tablet_manager.cc:1397] T ad7f0381ac1c425d952bced9f1f9f583 P 9557d17aae1b4c8c962a0fbb25048155: Time spent bootstrapping tablet: real 0.014s user 0.008s sys 0.000s
I20260811 09:12:50.554873 29533 tablet_bootstrap.cc:654] T ad7f0381ac1c425d952bced9f1f9f583 P e9b213925dca400bbb27881af693e926: Neither blocks nor log segments found. Creating new log.
I20260811 09:12:50.555615 29529 log.cc:824] T ad7f0381ac1c425d952bced9f1f9f583 P 4cabf1c01fa24edf81d1f561252ba26e: Log is configured to *not* fsync() on all Append() calls
I20260811 09:12:50.556275 29533 log.cc:824] T ad7f0381ac1c425d952bced9f1f9f583 P e9b213925dca400bbb27881af693e926: Log is configured to *not* fsync() on all Append() calls
I20260811 09:12:50.558772 29529 tablet_bootstrap.cc:492] T ad7f0381ac1c425d952bced9f1f9f583 P 4cabf1c01fa24edf81d1f561252ba26e: No bootstrap required, opened a new log
I20260811 09:12:50.559037 29529 ts_tablet_manager.cc:1397] T ad7f0381ac1c425d952bced9f1f9f583 P 4cabf1c01fa24edf81d1f561252ba26e: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260811 09:12:50.559458 29533 tablet_bootstrap.cc:492] T ad7f0381ac1c425d952bced9f1f9f583 P e9b213925dca400bbb27881af693e926: No bootstrap required, opened a new log
I20260811 09:12:50.559721 29533 ts_tablet_manager.cc:1397] T ad7f0381ac1c425d952bced9f1f9f583 P e9b213925dca400bbb27881af693e926: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260811 09:12:50.565310 29532 raft_consensus.cc:348] T ad7f0381ac1c425d952bced9f1f9f583 P 9557d17aae1b4c8c962a0fbb25048155 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cabf1c01fa24edf81d1f561252ba26e" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 36817 } } peers { permanent_uuid: "e9b213925dca400bbb27881af693e926" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 35071 } } peers { permanent_uuid: "9557d17aae1b4c8c962a0fbb25048155" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 35101 } }
I20260811 09:12:50.565831 29532 raft_consensus.cc:374] T ad7f0381ac1c425d952bced9f1f9f583 P 9557d17aae1b4c8c962a0fbb25048155 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:12:50.565986 29532 raft_consensus.cc:729] T ad7f0381ac1c425d952bced9f1f9f583 P 9557d17aae1b4c8c962a0fbb25048155 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9557d17aae1b4c8c962a0fbb25048155, State: Initialized, Role: FOLLOWER
I20260811 09:12:50.566699 29532 consensus_queue.cc:260] T ad7f0381ac1c425d952bced9f1f9f583 P 9557d17aae1b4c8c962a0fbb25048155 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cabf1c01fa24edf81d1f561252ba26e" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 36817 } } peers { permanent_uuid: "e9b213925dca400bbb27881af693e926" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 35071 } } peers { permanent_uuid: "9557d17aae1b4c8c962a0fbb25048155" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 35101 } }
I20260811 09:12:50.566965 29529 raft_consensus.cc:348] T ad7f0381ac1c425d952bced9f1f9f583 P 4cabf1c01fa24edf81d1f561252ba26e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cabf1c01fa24edf81d1f561252ba26e" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 36817 } } peers { permanent_uuid: "e9b213925dca400bbb27881af693e926" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 35071 } } peers { permanent_uuid: "9557d17aae1b4c8c962a0fbb25048155" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 35101 } }
I20260811 09:12:50.567476 29529 raft_consensus.cc:374] T ad7f0381ac1c425d952bced9f1f9f583 P 4cabf1c01fa24edf81d1f561252ba26e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:12:50.567634 29529 raft_consensus.cc:729] T ad7f0381ac1c425d952bced9f1f9f583 P 4cabf1c01fa24edf81d1f561252ba26e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cabf1c01fa24edf81d1f561252ba26e, State: Initialized, Role: FOLLOWER
I20260811 09:12:50.568423 29529 consensus_queue.cc:260] T ad7f0381ac1c425d952bced9f1f9f583 P 4cabf1c01fa24edf81d1f561252ba26e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cabf1c01fa24edf81d1f561252ba26e" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 36817 } } peers { permanent_uuid: "e9b213925dca400bbb27881af693e926" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 35071 } } peers { permanent_uuid: "9557d17aae1b4c8c962a0fbb25048155" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 35101 } }
I20260811 09:12:50.568660 29532 ts_tablet_manager.cc:1428] T ad7f0381ac1c425d952bced9f1f9f583 P 9557d17aae1b4c8c962a0fbb25048155: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260811 09:12:50.569808 29533 raft_consensus.cc:348] T ad7f0381ac1c425d952bced9f1f9f583 P e9b213925dca400bbb27881af693e926 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cabf1c01fa24edf81d1f561252ba26e" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 36817 } } peers { permanent_uuid: "e9b213925dca400bbb27881af693e926" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 35071 } } peers { permanent_uuid: "9557d17aae1b4c8c962a0fbb25048155" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 35101 } }
I20260811 09:12:50.570149 29533 raft_consensus.cc:374] T ad7f0381ac1c425d952bced9f1f9f583 P e9b213925dca400bbb27881af693e926 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:12:50.570241 29533 raft_consensus.cc:729] T ad7f0381ac1c425d952bced9f1f9f583 P e9b213925dca400bbb27881af693e926 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9b213925dca400bbb27881af693e926, State: Initialized, Role: FOLLOWER
I20260811 09:12:50.570434 29529 ts_tablet_manager.cc:1428] T ad7f0381ac1c425d952bced9f1f9f583 P 4cabf1c01fa24edf81d1f561252ba26e: Time spent starting tablet: real 0.011s user 0.008s sys 0.004s
I20260811 09:12:50.570773 29533 consensus_queue.cc:260] T ad7f0381ac1c425d952bced9f1f9f583 P e9b213925dca400bbb27881af693e926 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cabf1c01fa24edf81d1f561252ba26e" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 36817 } } peers { permanent_uuid: "e9b213925dca400bbb27881af693e926" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 35071 } } peers { permanent_uuid: "9557d17aae1b4c8c962a0fbb25048155" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 35101 } }
I20260811 09:12:50.570983 29247 heartbeater.cc:503] Master 127.28.154.190:38689 was elected leader, sending a full tablet report...
I20260811 09:12:50.571815 29503 heartbeater.cc:503] Master 127.28.154.190:38689 was elected leader, sending a full tablet report...
I20260811 09:12:50.573239 29533 ts_tablet_manager.cc:1428] T ad7f0381ac1c425d952bced9f1f9f583 P e9b213925dca400bbb27881af693e926: Time spent starting tablet: real 0.013s user 0.014s sys 0.000s
W20260811 09:12:50.602054 29259 tablet.cc:2369] T ad7f0381ac1c425d952bced9f1f9f583 P 4cabf1c01fa24edf81d1f561252ba26e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 09:12:50.656469 29540 raft_consensus.cc:482] T ad7f0381ac1c425d952bced9f1f9f583 P e9b213925dca400bbb27881af693e926 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:12:50.656796 29540 raft_consensus.cc:504] T ad7f0381ac1c425d952bced9f1f9f583 P e9b213925dca400bbb27881af693e926 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cabf1c01fa24edf81d1f561252ba26e" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 36817 } } peers { permanent_uuid: "e9b213925dca400bbb27881af693e926" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 35071 } } peers { permanent_uuid: "9557d17aae1b4c8c962a0fbb25048155" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 35101 } }
I20260811 09:12:50.658685 29540 leader_election.cc:288] T ad7f0381ac1c425d952bced9f1f9f583 P e9b213925dca400bbb27881af693e926 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4cabf1c01fa24edf81d1f561252ba26e (127.28.154.130:36817), 9557d17aae1b4c8c962a0fbb25048155 (127.28.154.129:35101)
W20260811 09:12:50.659825 29504 tablet.cc:2369] T ad7f0381ac1c425d952bced9f1f9f583 P e9b213925dca400bbb27881af693e926: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 09:12:50.665577 29521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:50.659266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.131:35271 (local address 127.28.154.130:36817)
0811 09:12:50.659463 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0811 09:12:50.659469 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:12:50.659707 (+ 238us) server_negotiation.cc:407] Connection header received
0811 09:12:50.659805 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:12:50.659810 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:12:50.659894 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:12:50.659995 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:12:50.660626 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:50.661614 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:50.663513 (+ 1899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:50.664698 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:50.664833 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:12:50.664897 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0811 09:12:50.665253 (+ 356us) server_negotiation.cc:299] Negotiation successful
0811 09:12:50.665352 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 09:12:50.665623 29507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:50.659176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.130:36817 (local address 127.28.154.131:35271)
0811 09:12:50.659476 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0811 09:12:50.659498 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 09:12:50.659674 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:12:50.660188 (+ 514us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:12:50.660193 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:12:50.660221 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:12:50.660455 (+ 234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:12:50.660468 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:12:50.661752 (+ 1284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:12:50.661758 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:12:50.663333 (+ 1575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:12:50.663348 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:12:50.664855 (+ 1507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:12:50.664862 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:12:50.665061 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:12:50.665122 (+ 61us) client_negotiation.cc:769] Sending connection context
0811 09:12:50.665217 (+ 95us) client_negotiation.cc:240] Negotiation successful
0811 09:12:50.665354 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141}
I20260811 09:12:50.666817 29526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:50.659241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.131:39391 (local address 127.28.154.129:35101)
0811 09:12:50.659642 (+ 401us) server_negotiation.cc:206] Beginning negotiation
0811 09:12:50.659647 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:12:50.660471 (+ 824us) server_negotiation.cc:407] Connection header received
0811 09:12:50.660712 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:12:50.660718 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:12:50.660797 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:12:50.660873 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:12:50.661464 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:50.662454 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:50.664630 (+ 2176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:50.665863 (+ 1233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:50.666014 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:12:50.666084 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 09:12:50.666459 (+ 375us) server_negotiation.cc:299] Negotiation successful
0811 09:12:50.666591 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":248}
I20260811 09:12:50.666903 29541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:50.659272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.129:35101 (local address 127.28.154.131:39391)
0811 09:12:50.660362 (+ 1090us) negotiation.cc:107] Waiting for socket to connect
0811 09:12:50.660384 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 09:12:50.660507 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:12:50.660922 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:12:50.660927 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:12:50.660955 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:12:50.661312 (+ 357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:12:50.661322 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:12:50.662632 (+ 1310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:12:50.662636 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:12:50.664439 (+ 1803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:12:50.664453 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:12:50.666077 (+ 1624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:12:50.666082 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:12:50.666276 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:12:50.666304 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 09:12:50.666415 (+ 111us) client_negotiation.cc:240] Negotiation successful
0811 09:12:50.666542 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":939,"mutex_wait_us":1,"thread_start_us":713,"threads_started":1}
I20260811 09:12:50.666903 29158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad7f0381ac1c425d952bced9f1f9f583" candidate_uuid: "e9b213925dca400bbb27881af693e926" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4cabf1c01fa24edf81d1f561252ba26e" is_pre_election: true
I20260811 09:12:50.667519 29158 raft_consensus.cc:2454] T ad7f0381ac1c425d952bced9f1f9f583 P 4cabf1c01fa24edf81d1f561252ba26e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9b213925dca400bbb27881af693e926 in term 0.
I20260811 09:12:50.668004 28927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad7f0381ac1c425d952bced9f1f9f583" candidate_uuid: "e9b213925dca400bbb27881af693e926" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9557d17aae1b4c8c962a0fbb25048155" is_pre_election: true
I20260811 09:12:50.668365 29302 leader_election.cc:302] T ad7f0381ac1c425d952bced9f1f9f583 P e9b213925dca400bbb27881af693e926 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4cabf1c01fa24edf81d1f561252ba26e, e9b213925dca400bbb27881af693e926; no voters:
I20260811 09:12:50.668537 28927 raft_consensus.cc:2454] T ad7f0381ac1c425d952bced9f1f9f583 P 9557d17aae1b4c8c962a0fbb25048155 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9b213925dca400bbb27881af693e926 in term 0.
I20260811 09:12:50.668787 29540 raft_consensus.cc:2781] T ad7f0381ac1c425d952bced9f1f9f583 P e9b213925dca400bbb27881af693e926 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 09:12:50.668943 29540 raft_consensus.cc:482] T ad7f0381ac1c425d952bced9f1f9f583 P e9b213925dca400bbb27881af693e926 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 09:12:50.669054 29540 raft_consensus.cc:3037] T ad7f0381ac1c425d952bced9f1f9f583 P e9b213925dca400bbb27881af693e926 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:12:50.671260 29540 raft_consensus.cc:504] T ad7f0381ac1c425d952bced9f1f9f583 P e9b213925dca400bbb27881af693e926 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cabf1c01fa24edf81d1f561252ba26e" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 36817 } } peers { permanent_uuid: "e9b213925dca400bbb27881af693e926" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 35071 } } peers { permanent_uuid: "9557d17aae1b4c8c962a0fbb25048155" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 35101 } }
I20260811 09:12:50.671841 29540 leader_election.cc:288] T ad7f0381ac1c425d952bced9f1f9f583 P e9b213925dca400bbb27881af693e926 [CANDIDATE]: Term 1 election: Requested vote from peers 4cabf1c01fa24edf81d1f561252ba26e (127.28.154.130:36817), 9557d17aae1b4c8c962a0fbb25048155 (127.28.154.129:35101)
I20260811 09:12:50.672472 28927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad7f0381ac1c425d952bced9f1f9f583" candidate_uuid: "e9b213925dca400bbb27881af693e926" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9557d17aae1b4c8c962a0fbb25048155"
I20260811 09:12:50.672508 29158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad7f0381ac1c425d952bced9f1f9f583" candidate_uuid: "e9b213925dca400bbb27881af693e926" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4cabf1c01fa24edf81d1f561252ba26e"
I20260811 09:12:50.672669 28927 raft_consensus.cc:3037] T ad7f0381ac1c425d952bced9f1f9f583 P 9557d17aae1b4c8c962a0fbb25048155 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:12:50.672747 29158 raft_consensus.cc:3037] T ad7f0381ac1c425d952bced9f1f9f583 P 4cabf1c01fa24edf81d1f561252ba26e [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:12:50.675079 29158 raft_consensus.cc:2454] T ad7f0381ac1c425d952bced9f1f9f583 P 4cabf1c01fa24edf81d1f561252ba26e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9b213925dca400bbb27881af693e926 in term 1.
I20260811 09:12:50.675191 28927 raft_consensus.cc:2454] T ad7f0381ac1c425d952bced9f1f9f583 P 9557d17aae1b4c8c962a0fbb25048155 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9b213925dca400bbb27881af693e926 in term 1.
I20260811 09:12:50.675612 29302 leader_election.cc:302] T ad7f0381ac1c425d952bced9f1f9f583 P e9b213925dca400bbb27881af693e926 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4cabf1c01fa24edf81d1f561252ba26e, e9b213925dca400bbb27881af693e926; no voters:
I20260811 09:12:50.675978 29540 raft_consensus.cc:2781] T ad7f0381ac1c425d952bced9f1f9f583 P e9b213925dca400bbb27881af693e926 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:12:50.676460 29540 raft_consensus.cc:686] T ad7f0381ac1c425d952bced9f1f9f583 P e9b213925dca400bbb27881af693e926 [term 1 LEADER]: Becoming Leader. State: Replica: e9b213925dca400bbb27881af693e926, State: Running, Role: LEADER
I20260811 09:12:50.676966 29540 consensus_queue.cc:237] T ad7f0381ac1c425d952bced9f1f9f583 P e9b213925dca400bbb27881af693e926 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cabf1c01fa24edf81d1f561252ba26e" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 36817 } } peers { permanent_uuid: "e9b213925dca400bbb27881af693e926" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 35071 } } peers { permanent_uuid: "9557d17aae1b4c8c962a0fbb25048155" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 35101 } }
I20260811 09:12:50.682118 28656 catalog_manager.cc:5495] T ad7f0381ac1c425d952bced9f1f9f583 P e9b213925dca400bbb27881af693e926 reported cstate change: term changed from 0 to 1, leader changed from <none> to e9b213925dca400bbb27881af693e926 (127.28.154.131). New cstate: current_term: 1 leader_uuid: "e9b213925dca400bbb27881af693e926" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4cabf1c01fa24edf81d1f561252ba26e" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 36817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9b213925dca400bbb27881af693e926" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 35071 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9557d17aae1b4c8c962a0fbb25048155" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 35101 } health_report { overall_health: UNKNOWN } } }
I20260811 09:12:50.719308 29527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:50.713446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54706 (local address 127.28.154.131:35071)
0811 09:12:50.713698 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0811 09:12:50.713705 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:12:50.713776 (+ 71us) server_negotiation.cc:407] Connection header received
0811 09:12:50.714003 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:12:50.714008 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:12:50.714104 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:12:50.714244 (+ 140us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:12:50.714640 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:50.715377 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:50.716814 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:50.717338 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:50.717471 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:12:50.718055 (+ 584us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:12:50.718596 (+ 541us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:12:50.718826 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0811 09:12:50.718944 (+ 118us) server_negotiation.cc:299] Negotiation successful
0811 09:12:50.719038 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 09:12:50.727968 28927 raft_consensus.cc:1261] T ad7f0381ac1c425d952bced9f1f9f583 P 9557d17aae1b4c8c962a0fbb25048155 [term 1 FOLLOWER]: Refusing update from remote peer e9b213925dca400bbb27881af693e926: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:12:50.727970 29158 raft_consensus.cc:1261] T ad7f0381ac1c425d952bced9f1f9f583 P 4cabf1c01fa24edf81d1f561252ba26e [term 1 FOLLOWER]: Refusing update from remote peer e9b213925dca400bbb27881af693e926: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:12:50.729138 29543 consensus_queue.cc:1035] T ad7f0381ac1c425d952bced9f1f9f583 P e9b213925dca400bbb27881af693e926 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9557d17aae1b4c8c962a0fbb25048155" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 35101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:12:50.729600 29540 consensus_queue.cc:1035] T ad7f0381ac1c425d952bced9f1f9f583 P e9b213925dca400bbb27881af693e926 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4cabf1c01fa24edf81d1f561252ba26e" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 36817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:12:50.758309 28656 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60798:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:12:50.758684 28656 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60798:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:12:50.763235 28656 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0cf5553defee462ab9c66631e7042a74: Sending DeleteTablet for 3 replicas of tablet ad7f0381ac1c425d952bced9f1f9f583
I20260811 09:12:50.764671 29375 tablet_service.cc:1514] Processing DeleteTablet for tablet ad7f0381ac1c425d952bced9f1f9f583 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:12:50 UTC) from {username='kudu', principal='kudu/127.28.154.190@KRBTEST.COM'} at 127.0.0.1:54700
I20260811 09:12:50.764667 28896 tablet_service.cc:1514] Processing DeleteTablet for tablet ad7f0381ac1c425d952bced9f1f9f583 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:12:50 UTC) from {username='kudu', principal='kudu/127.28.154.190@KRBTEST.COM'} at 127.0.0.1:57974
I20260811 09:12:50.764914 29124 tablet_service.cc:1514] Processing DeleteTablet for tablet ad7f0381ac1c425d952bced9f1f9f583 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:12:50 UTC) from {username='kudu', principal='kudu/127.28.154.190@KRBTEST.COM'} at 127.0.0.1:40632
I20260811 09:12:50.767628 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 28770
I20260811 09:12:50.774274 29567 tablet_replica.cc:330] T ad7f0381ac1c425d952bced9f1f9f583 P 9557d17aae1b4c8c962a0fbb25048155: stopping tablet replica
I20260811 09:12:50.775161 29565 tablet_replica.cc:330] T ad7f0381ac1c425d952bced9f1f9f583 P 4cabf1c01fa24edf81d1f561252ba26e: stopping tablet replica
I20260811 09:12:50.775640 29565 raft_consensus.cc:2229] T ad7f0381ac1c425d952bced9f1f9f583 P 4cabf1c01fa24edf81d1f561252ba26e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:12:50.776126 29567 raft_consensus.cc:2229] T ad7f0381ac1c425d952bced9f1f9f583 P 9557d17aae1b4c8c962a0fbb25048155 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:12:50.776294 29565 raft_consensus.cc:2258] T ad7f0381ac1c425d952bced9f1f9f583 P 4cabf1c01fa24edf81d1f561252ba26e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:12:50.777215 29566 tablet_replica.cc:330] T ad7f0381ac1c425d952bced9f1f9f583 P e9b213925dca400bbb27881af693e926: stopping tablet replica
I20260811 09:12:50.777666 29566 raft_consensus.cc:2229] T ad7f0381ac1c425d952bced9f1f9f583 P e9b213925dca400bbb27881af693e926 [term 1 LEADER]: Raft consensus shutting down.
I20260811 09:12:50.778438 29566 raft_consensus.cc:2258] T ad7f0381ac1c425d952bced9f1f9f583 P e9b213925dca400bbb27881af693e926 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:12:50.779860 29565 ts_tablet_manager.cc:1905] T ad7f0381ac1c425d952bced9f1f9f583 P 4cabf1c01fa24edf81d1f561252ba26e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:12:50.780598 29566 ts_tablet_manager.cc:1905] T ad7f0381ac1c425d952bced9f1f9f583 P e9b213925dca400bbb27881af693e926: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:12:50.776978 29567 raft_consensus.cc:2258] T ad7f0381ac1c425d952bced9f1f9f583 P 9557d17aae1b4c8c962a0fbb25048155 [term 1 FOLLOWER]: Raft consensus is shut down!
Aug 11 09:12:50 dist-test-slave-xt4k krb5kdc[28580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439568, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.154.129@KRBTEST.COM
I20260811 09:12:50.787667 29565 ts_tablet_manager.cc:1918] T ad7f0381ac1c425d952bced9f1f9f583 P 4cabf1c01fa24edf81d1f561252ba26e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:12:50.787985 29565 log.cc:1196] T ad7f0381ac1c425d952bced9f1f9f583 P 4cabf1c01fa24edf81d1f561252ba26e: Deleting WAL directory at /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439250192320-29290-0/minicluster-data/ts-1/wal/wals/ad7f0381ac1c425d952bced9f1f9f583
I20260811 09:12:50.788682 29565 ts_tablet_manager.cc:1939] T ad7f0381ac1c425d952bced9f1f9f583 P 4cabf1c01fa24edf81d1f561252ba26e: Deleting consensus metadata
I20260811 09:12:50.789754 29566 ts_tablet_manager.cc:1918] T ad7f0381ac1c425d952bced9f1f9f583 P e9b213925dca400bbb27881af693e926: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:12:50.790028 29566 log.cc:1196] T ad7f0381ac1c425d952bced9f1f9f583 P e9b213925dca400bbb27881af693e926: Deleting WAL directory at /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786439250192320-29290-0/minicluster-data/ts-2/wal/wals/ad7f0381ac1c425d952bced9f1f9f583
I20260811 09:12:50.790709 29566 ts_tablet_manager.cc:1939] T ad7f0381ac1c425d952bced9f1f9f583 P e9b213925dca400bbb27881af693e926: Deleting consensus metadata
I20260811 09:12:50.790995 28641 catalog_manager.cc:4841] TS 4cabf1c01fa24edf81d1f561252ba26e (127.28.154.130:36817): tablet ad7f0381ac1c425d952bced9f1f9f583 (table test-table [id=c833994cbfe7410492d2a2ea33cc7b2e]) successfully deleted
I20260811 09:12:50.792882 28641 catalog_manager.cc:4841] TS e9b213925dca400bbb27881af693e926 (127.28.154.131:35071): tablet ad7f0381ac1c425d952bced9f1f9f583 (table test-table [id=c833994cbfe7410492d2a2ea33cc7b2e]) successfully deleted
I20260811 09:12:50.793318 29567 ts_tablet_manager.cc:1905] T ad7f0381ac1c425d952bced9f1f9f583 P 9557d17aae1b4c8c962a0fbb25048155: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:12:50.794806 29526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:50.769227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57982 (local address 127.28.154.129:35101)
0811 09:12:50.769432 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0811 09:12:50.769438 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:12:50.774154 (+ 4716us) server_negotiation.cc:407] Connection header received
0811 09:12:50.774368 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:12:50.774375 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:12:50.774484 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:12:50.774599 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:12:50.776733 (+ 2134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:50.777926 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:50.779889 (+ 1963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:50.780540 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:50.780678 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:12:50.786774 (+ 6096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:12:50.786805 (+ 31us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:12:50.786830 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:12:50.786886 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:12:50.790976 (+ 4090us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:12:50.792100 (+ 1124us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:12:50.792113 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:12:50.792116 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:12:50.792313 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:12:50.792830 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:12:50.792836 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:12:50.792839 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:12:50.793351 (+ 512us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:12:50.793613 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0811 09:12:50.794159 (+ 546us) server_negotiation.cc:299] Negotiation successful
0811 09:12:50.794388 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 09:12:50.795537 28998 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:12:50.929531 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 28770
W20260811 09:12:50.956174 28642 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.28.154.129:35101 (error 108)
I20260811 09:12:50.957100 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 29021
I20260811 09:12:50.958397 29523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:50.957798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.129:35101 (local address 127.0.0.1:57994)
0811 09:12:50.958079 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0811 09:12:50.958213 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.129:35101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159}
W20260811 09:12:50.958669 28642 catalog_manager.cc:4584] TS 9557d17aae1b4c8c962a0fbb25048155 (127.28.154.129:35101): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ad7f0381ac1c425d952bced9f1f9f583: Network error: Client connection negotiation failed: client connection to 127.28.154.129:35101: connect: Connection refused (error 111)
Aug 11 09:12:50 dist-test-slave-xt4k krb5kdc[28580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439568, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.154.130@KRBTEST.COM
I20260811 09:12:50.975380 29521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:50.957957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40648 (local address 127.28.154.130:36817)
0811 09:12:50.958174 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0811 09:12:50.958180 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:12:50.958201 (+ 21us) server_negotiation.cc:407] Connection header received
0811 09:12:50.958382 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:12:50.958388 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:12:50.958473 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:12:50.958574 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:12:50.960072 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:50.961263 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:50.962818 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:50.963445 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:50.963573 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:12:50.968044 (+ 4471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:12:50.968066 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:12:50.968089 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:12:50.968144 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:12:50.972164 (+ 4020us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:12:50.973094 (+ 930us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:12:50.973104 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:12:50.973108 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:12:50.973273 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:12:50.973618 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:12:50.973623 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:12:50.973626 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:12:50.974154 (+ 528us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:12:50.974448 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0811 09:12:50.974784 (+ 336us) server_negotiation.cc:299] Negotiation successful
0811 09:12:50.975027 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260811 09:12:50.976130 29239 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:12:51.012796 29523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:51.012137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.129:35101 (local address 127.0.0.1:58004)
0811 09:12:51.012455 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0811 09:12:51.012585 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.129:35101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117}
I20260811 09:12:51.014145 29523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:51.013690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.129:35101 (local address 127.0.0.1:58014)
0811 09:12:51.013862 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0811 09:12:51.013961 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.129:35101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260811 09:12:51.069315 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 29021
I20260811 09:12:51.077194 29523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:51.076689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.129:35101 (local address 127.0.0.1:58022)
0811 09:12:51.076947 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0811 09:12:51.077032 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.129:35101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144}
I20260811 09:12:51.089627 29523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:51.089062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.129:35101 (local address 127.0.0.1:58038)
0811 09:12:51.089327 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0811 09:12:51.089434 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.129:35101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260811 09:12:51.101823 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 29266
Aug 11 09:12:51 dist-test-slave-xt4k krb5kdc[28580](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439568, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.154.131@KRBTEST.COM
I20260811 09:12:51.121801 29527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:51.102708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54718 (local address 127.28.154.131:35071)
0811 09:12:51.102898 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 09:12:51.102903 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:12:51.102923 (+ 20us) server_negotiation.cc:407] Connection header received
0811 09:12:51.103008 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:12:51.103012 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:12:51.103074 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:12:51.103172 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:12:51.105083 (+ 1911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:51.105900 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:51.107725 (+ 1825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:51.108248 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:51.108410 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:12:51.114142 (+ 5732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:12:51.114167 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:12:51.114192 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:12:51.114260 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:12:51.118416 (+ 4156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:12:51.119282 (+ 866us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:12:51.119293 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:12:51.119297 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:12:51.119514 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:12:51.119966 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:12:51.119972 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:12:51.119975 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:12:51.120543 (+ 568us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:12:51.120900 (+ 357us) server_negotiation.cc:1035] Waiting for connection context
0811 09:12:51.121159 (+ 259us) server_negotiation.cc:299] Negotiation successful
0811 09:12:51.121407 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 09:12:51.122721 29493 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:12:51.170555 29523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:51.170010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.129:35101 (local address 127.0.0.1:58052)
0811 09:12:51.170254 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0811 09:12:51.170357 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.129:35101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260811 09:12:51.171762 29523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:51.171335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.129:35101 (local address 127.0.0.1:58058)
0811 09:12:51.171537 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0811 09:12:51.171641 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.129:35101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
I20260811 09:12:51.251469 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 29266
I20260811 09:12:51.279165 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 28602
I20260811 09:12:51.280599 28714 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:12:51.398736 29523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:51.398247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.129:35101 (local address 127.0.0.1:58062)
0811 09:12:51.398451 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0811 09:12:51.398551 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.129:35101: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260811 09:12:51.399165 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 28602
2026-08-11T09:12:51Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (4863 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-11T09:12:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:12:51Z Disabled control of system clock
I20260811 09:12:51.503599 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439250192320-29290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439250192320-29290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439250192320-29290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.154.190:33749
--webserver_interface=127.28.154.190
--webserver_port=0
--builtin_ntp_servers=127.28.154.148:39777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.154.190:33749
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 09:12:51.872428 29584 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:12:51.872736 29584 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:12:51.872828 29584 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:12:51.880903 29584 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:12:51.881054 29584 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:12:51.881100 29584 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:12:51.881174 29584 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:12:51.881220 29584 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:12:51.890977 29584 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:39777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439250192320-29290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439250192320-29290-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.154.190:33749
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.154.190:33749
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.154.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439250192320-29290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:12:51.892279 29584 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:12:51.894160 29584 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:12:51.906785 29594 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:12:51.906870 29590 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:12:51.906821 29589 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:12:51.907846 29584 server_base.cc:1034] running on GCE node
I20260811 09:12:51.908797 29584 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:12:51.910398 29584 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:12:51.911631 29584 hybrid_clock.cc:648] HybridClock initialized: now 1786439571911573 us; error 45 us; skew 500 ppm
I20260811 09:12:51.912014 29584 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 09:12:51.918979 29584 webserver.cc:460] Webserver started at http://127.28.154.190:45925/ using document root <none> and password file <none>
I20260811 09:12:51.919831 29584 fs_manager.cc:359] Metadata directory not provided
I20260811 09:12:51.919941 29584 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:12:51.920287 29584 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:12:51.923413 29584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439250192320-29290-0/minicluster-data/master-0/data/instance:
uuid: "b435685007cc4b67a6edef334c0ba915"
format_stamp: "Formatted at 2026-08-11 09:12:51 on dist-test-slave-xt4k"
server_key: "461d8abc4c34c9b2df15859998af6d84"
server_key_iv: "834c961880851d8070e6fa6bf9d1db0d"
server_key_version: "encryptionkey@0"
I20260811 09:12:51.924255 29584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439250192320-29290-0/minicluster-data/master-0/wal/instance:
uuid: "b435685007cc4b67a6edef334c0ba915"
format_stamp: "Formatted at 2026-08-11 09:12:51 on dist-test-slave-xt4k"
server_key: "461d8abc4c34c9b2df15859998af6d84"
server_key_iv: "834c961880851d8070e6fa6bf9d1db0d"
server_key_version: "encryptionkey@0"
I20260811 09:12:51.929837 29584 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260811 09:12:51.933804 29610 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:12:51.935524 29584 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.002s
I20260811 09:12:51.935736 29584 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439250192320-29290-0/minicluster-data/master-0/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439250192320-29290-0/minicluster-data/master-0/wal
uuid: "b435685007cc4b67a6edef334c0ba915"
format_stamp: "Formatted at 2026-08-11 09:12:51 on dist-test-slave-xt4k"
server_key: "461d8abc4c34c9b2df15859998af6d84"
server_key_iv: "834c961880851d8070e6fa6bf9d1db0d"
server_key_version: "encryptionkey@0"
I20260811 09:12:51.935971 29584 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439250192320-29290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439250192320-29290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439250192320-29290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:12:51.971035 29584 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:12:51.971966 29584 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:12:51.972255 29584 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:12:51.990834 29584 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.190:33749
I20260811 09:12:51.990859 29704 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.190:33749 every 8 connection(s)
I20260811 09:12:51.992223 29584 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
I20260811 09:12:51.996133 29707 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:12:51.998804 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 29584
I20260811 09:12:51.998999 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439250192320-29290-0/minicluster-data/master-0/wal/instance
I20260811 09:12:51.999571 29290 external_mini_cluster.cc:1428] Setting key 6c37a096661ee398f53fafb3b28547ae
I20260811 09:12:52.009615 29707 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b435685007cc4b67a6edef334c0ba915: Bootstrap starting.
I20260811 09:12:52.013239 29707 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b435685007cc4b67a6edef334c0ba915: Neither blocks nor log segments found. Creating new log.
I20260811 09:12:52.014482 29707 log.cc:824] T 00000000000000000000000000000000 P b435685007cc4b67a6edef334c0ba915: Log is configured to *not* fsync() on all Append() calls
I20260811 09:12:52.015986 29712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:52.001917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44124 (local address 127.28.154.190:33749)
0811 09:12:52.003251 (+ 1334us) server_negotiation.cc:206] Beginning negotiation
0811 09:12:52.003263 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 09:12:52.003310 (+ 47us) server_negotiation.cc:407] Connection header received
0811 09:12:52.005340 (+ 2030us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:12:52.005367 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:12:52.005905 (+ 538us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:12:52.006414 (+ 509us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:12:52.007797 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:52.009199 (+ 1402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:52.010784 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:52.011544 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:52.011740 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:12:52.013031 (+ 1291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:12:52.013040 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:12:52.013058 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:12:52.013721 (+ 663us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:12:52.013838 (+ 117us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:12:52.013862 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:12:52.013975 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 09:12:52.014216 (+ 241us) server_negotiation.cc:299] Negotiation successful
0811 09:12:52.014565 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":854,"thread_start_us":478,"threads_started":1}
I20260811 09:12:52.018206 29707 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b435685007cc4b67a6edef334c0ba915: No bootstrap required, opened a new log
I20260811 09:12:52.026229 29707 raft_consensus.cc:348] T 00000000000000000000000000000000 P b435685007cc4b67a6edef334c0ba915 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b435685007cc4b67a6edef334c0ba915" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 33749 } }
I20260811 09:12:52.026644 29707 raft_consensus.cc:374] T 00000000000000000000000000000000 P b435685007cc4b67a6edef334c0ba915 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:12:52.026741 29707 raft_consensus.cc:729] T 00000000000000000000000000000000 P b435685007cc4b67a6edef334c0ba915 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b435685007cc4b67a6edef334c0ba915, State: Initialized, Role: FOLLOWER
I20260811 09:12:52.027333 29707 consensus_queue.cc:260] T 00000000000000000000000000000000 P b435685007cc4b67a6edef334c0ba915 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b435685007cc4b67a6edef334c0ba915" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 33749 } }
I20260811 09:12:52.027606 29707 raft_consensus.cc:388] T 00000000000000000000000000000000 P b435685007cc4b67a6edef334c0ba915 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:12:52.027740 29707 raft_consensus.cc:482] T 00000000000000000000000000000000 P b435685007cc4b67a6edef334c0ba915 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:12:52.027904 29707 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b435685007cc4b67a6edef334c0ba915 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:12:52.030469 29707 raft_consensus.cc:504] T 00000000000000000000000000000000 P b435685007cc4b67a6edef334c0ba915 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b435685007cc4b67a6edef334c0ba915" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 33749 } }
I20260811 09:12:52.031009 29707 leader_election.cc:302] T 00000000000000000000000000000000 P b435685007cc4b67a6edef334c0ba915 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b435685007cc4b67a6edef334c0ba915; no voters:
I20260811 09:12:52.031616 29707 leader_election.cc:288] T 00000000000000000000000000000000 P b435685007cc4b67a6edef334c0ba915 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:12:52.031756 29716 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b435685007cc4b67a6edef334c0ba915 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:12:52.032464 29716 raft_consensus.cc:686] T 00000000000000000000000000000000 P b435685007cc4b67a6edef334c0ba915 [term 1 LEADER]: Becoming Leader. State: Replica: b435685007cc4b67a6edef334c0ba915, State: Running, Role: LEADER
I20260811 09:12:52.032987 29716 consensus_queue.cc:237] T 00000000000000000000000000000000 P b435685007cc4b67a6edef334c0ba915 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b435685007cc4b67a6edef334c0ba915" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 33749 } }
I20260811 09:12:52.033782 29707 sys_catalog.cc:564] T 00000000000000000000000000000000 P b435685007cc4b67a6edef334c0ba915 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:12:52.035848 29718 sys_catalog.cc:455] T 00000000000000000000000000000000 P b435685007cc4b67a6edef334c0ba915 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b435685007cc4b67a6edef334c0ba915" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b435685007cc4b67a6edef334c0ba915" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 33749 } } }
I20260811 09:12:52.036376 29718 sys_catalog.cc:458] T 00000000000000000000000000000000 P b435685007cc4b67a6edef334c0ba915 [sys.catalog]: This master's current role is: LEADER
I20260811 09:12:52.035821 29719 sys_catalog.cc:455] T 00000000000000000000000000000000 P b435685007cc4b67a6edef334c0ba915 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b435685007cc4b67a6edef334c0ba915. Latest consensus state: current_term: 1 leader_uuid: "b435685007cc4b67a6edef334c0ba915" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b435685007cc4b67a6edef334c0ba915" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 33749 } } }
I20260811 09:12:52.036592 29719 sys_catalog.cc:458] T 00000000000000000000000000000000 P b435685007cc4b67a6edef334c0ba915 [sys.catalog]: This master's current role is: LEADER
I20260811 09:12:52.051100 29729 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:12:52.056320 29729 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:12:52.066056 29729 catalog_manager.cc:1347] Generated new cluster ID: 2fabe8661e324d87ba9a6cc535e909d3
I20260811 09:12:52.066185 29729 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:12:52.089478 29729 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:12:52.091460 29729 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:12:52.104794 29729 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b435685007cc4b67a6edef334c0ba915: Generated new TSK 0
I20260811 09:12:52.106163 29729 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 09:12:52.120554 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439250192320-29290-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439250192320-29290-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439250192320-29290-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439250192320-29290-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.154.129:0
--local_ip_for_outbound_sockets=127.28.154.129
--webserver_interface=127.28.154.129
--webserver_port=0
--tserver_master_addrs=127.28.154.190:33749
--builtin_ntp_servers=127.28.154.148:39777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 09:12:52.486168 29760 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:12:52.486490 29760 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:12:52.486615 29760 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:12:52.494153 29760 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:12:52.494277 29760 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:12:52.494449 29760 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.154.129
I20260811 09:12:52.504611 29760 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:39777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439250192320-29290-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439250192320-29290-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.154.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439250192320-29290-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.154.129
--webserver_port=0
--tserver_master_addrs=127.28.154.190:33749
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.154.129
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439250192320-29290-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:12:52.505967 29760 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:12:52.507782 29760 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:12:52.521376 29769 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:12:52.521349 29773 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:12:52.521559 29779 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:12:52.522136 29760 server_base.cc:1034] running on GCE node
I20260811 09:12:52.522758 29760 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:12:52.523738 29760 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:12:52.524945 29760 hybrid_clock.cc:648] HybridClock initialized: now 1786439572524895 us; error 37 us; skew 500 ppm
I20260811 09:12:52.525269 29760 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 09:12:52.527534 29760 webserver.cc:460] Webserver started at http://127.28.154.129:37165/ using document root <none> and password file <none>
I20260811 09:12:52.528252 29760 fs_manager.cc:359] Metadata directory not provided
I20260811 09:12:52.528388 29760 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:12:52.528723 29760 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:12:52.531608 29760 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439250192320-29290-0/minicluster-data/ts-0/data/instance:
uuid: "94f08e6fe92a433c986fb6872f4da8bc"
format_stamp: "Formatted at 2026-08-11 09:12:52 on dist-test-slave-xt4k"
server_key: "3f6148f5b5cb6487909fa04b20c8981f"
server_key_iv: "b3aa868187c15b4bd0877b60b562bbfa"
server_key_version: "encryptionkey@0"
I20260811 09:12:52.532426 29760 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439250192320-29290-0/minicluster-data/ts-0/wal/instance:
uuid: "94f08e6fe92a433c986fb6872f4da8bc"
format_stamp: "Formatted at 2026-08-11 09:12:52 on dist-test-slave-xt4k"
server_key: "3f6148f5b5cb6487909fa04b20c8981f"
server_key_iv: "b3aa868187c15b4bd0877b60b562bbfa"
server_key_version: "encryptionkey@0"
I20260811 09:12:52.537904 29760 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260811 09:12:52.541827 29786 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:12:52.543282 29760 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260811 09:12:52.543447 29760 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439250192320-29290-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439250192320-29290-0/minicluster-data/ts-0/wal
uuid: "94f08e6fe92a433c986fb6872f4da8bc"
format_stamp: "Formatted at 2026-08-11 09:12:52 on dist-test-slave-xt4k"
server_key: "3f6148f5b5cb6487909fa04b20c8981f"
server_key_iv: "b3aa868187c15b4bd0877b60b562bbfa"
server_key_version: "encryptionkey@0"
I20260811 09:12:52.543677 29760 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439250192320-29290-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439250192320-29290-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439250192320-29290-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:12:52.574815 29760 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:12:52.575733 29760 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:12:52.576048 29760 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:12:52.577096 29760 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:12:52.579097 29760 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:12:52.579353 29760 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:12:52.579501 29760 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:12:52.579576 29760 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:12:52.611311 29760 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.129:33867
I20260811 09:12:52.611374 29991 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.129:33867 every 8 connection(s)
I20260811 09:12:52.612687 29760 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439250192320-29290-0/minicluster-data/ts-0/data/info.pb
I20260811 09:12:52.615810 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 29760
I20260811 09:12:52.615971 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439250192320-29290-0/minicluster-data/ts-0/wal/instance
I20260811 09:12:52.616559 29290 external_mini_cluster.cc:1428] Setting key 154b62df9fe14eadbab58a610ae2b235
I20260811 09:12:52.619536 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439250192320-29290-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439250192320-29290-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439250192320-29290-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439250192320-29290-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.154.130:0
--local_ip_for_outbound_sockets=127.28.154.130
--webserver_interface=127.28.154.130
--webserver_port=0
--tserver_master_addrs=127.28.154.190:33749
--builtin_ntp_servers=127.28.154.148:39777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 09:12:52.626464 29996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:52.615179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.129:51353 (local address 127.28.154.190:33749)
0811 09:12:52.615681 (+ 502us) server_negotiation.cc:206] Beginning negotiation
0811 09:12:52.615688 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:12:52.617131 (+ 1443us) server_negotiation.cc:407] Connection header received
0811 09:12:52.619030 (+ 1899us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:12:52.619034 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:12:52.619115 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:12:52.619197 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:12:52.620611 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:52.621328 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:52.623406 (+ 2078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:52.623830 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:52.623958 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:12:52.625630 (+ 1672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:12:52.625636 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:12:52.625639 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:12:52.625660 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:12:52.625709 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:12:52.625722 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:12:52.625798 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 09:12:52.626096 (+ 298us) server_negotiation.cc:299] Negotiation successful
0811 09:12:52.626190 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":230,"threads_started":1}
I20260811 09:12:52.627802 29997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:52.615808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:33749 (local address 127.28.154.129:51353)
0811 09:12:52.616943 (+ 1135us) negotiation.cc:107] Waiting for socket to connect
0811 09:12:52.617002 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 09:12:52.618720 (+ 1718us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:12:52.619471 (+ 751us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:12:52.619483 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:12:52.619605 (+ 122us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:12:52.620429 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:12:52.620456 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:12:52.621480 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:12:52.621485 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:12:52.623246 (+ 1761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:12:52.623261 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:12:52.624092 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:12:52.624100 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:12:52.624465 (+ 365us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:12:52.625253 (+ 788us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 09:12:52.625256 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:12:52.625311 (+ 55us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 09:12:52.625415 (+ 104us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:12:52.625820 (+ 405us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:12:52.625831 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:12:52.625838 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 09:12:52.626017 (+ 179us) client_negotiation.cc:240] Negotiation successful
0811 09:12:52.626437 (+ 420us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":536,"thread_start_us":297,"threads_started":1}
I20260811 09:12:52.629398 29992 heartbeater.cc:347] Connected to a master server at 127.28.154.190:33749
I20260811 09:12:52.629727 29992 heartbeater.cc:464] Registering TS with master...
I20260811 09:12:52.630502 29992 heartbeater.cc:511] Master 127.28.154.190:33749 requested a full tablet report, sending...
I20260811 09:12:52.632668 29635 ts_manager.cc:194] Registered new tserver with Master: 94f08e6fe92a433c986fb6872f4da8bc (127.28.154.129:33867)
I20260811 09:12:52.634451 29635 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.28.154.129:51353
W20260811 09:12:52.974514 30000 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:12:52.974957 30000 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:12:52.975121 30000 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:12:52.982934 30000 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:12:52.983070 30000 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:12:52.983210 30000 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.154.130
I20260811 09:12:52.992606 30000 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:39777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439250192320-29290-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439250192320-29290-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.154.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439250192320-29290-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.154.130
--webserver_port=0
--tserver_master_addrs=127.28.154.190:33749
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.154.130
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439250192320-29290-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:12:52.993913 30000 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:12:52.996042 30000 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:12:53.010300 30008 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:12:53.010735 30012 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:12:53.011270 30009 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:12:53.011718 30000 server_base.cc:1034] running on GCE node
I20260811 09:12:53.012470 30000 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:12:53.013523 30000 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:12:53.014868 30000 hybrid_clock.cc:648] HybridClock initialized: now 1786439573014826 us; error 43 us; skew 500 ppm
I20260811 09:12:53.015349 30000 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 09:12:53.019200 30000 webserver.cc:460] Webserver started at http://127.28.154.130:38027/ using document root <none> and password file <none>
I20260811 09:12:53.020174 30000 fs_manager.cc:359] Metadata directory not provided
I20260811 09:12:53.020299 30000 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:12:53.020697 30000 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:12:53.023916 30000 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439250192320-29290-0/minicluster-data/ts-1/data/instance:
uuid: "c2f51e4313c5403aa360293afbb6b0fd"
format_stamp: "Formatted at 2026-08-11 09:12:53 on dist-test-slave-xt4k"
server_key: "ff3b9f27bcfa828e6aa29abaf4356386"
server_key_iv: "4e48c375a29988a27f37818e7e4c3937"
server_key_version: "encryptionkey@0"
I20260811 09:12:53.025004 30000 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439250192320-29290-0/minicluster-data/ts-1/wal/instance:
uuid: "c2f51e4313c5403aa360293afbb6b0fd"
format_stamp: "Formatted at 2026-08-11 09:12:53 on dist-test-slave-xt4k"
server_key: "ff3b9f27bcfa828e6aa29abaf4356386"
server_key_iv: "4e48c375a29988a27f37818e7e4c3937"
server_key_version: "encryptionkey@0"
I20260811 09:12:53.030851 30000 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260811 09:12:53.034950 30022 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:12:53.036533 30000 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 09:12:53.036711 30000 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439250192320-29290-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439250192320-29290-0/minicluster-data/ts-1/wal
uuid: "c2f51e4313c5403aa360293afbb6b0fd"
format_stamp: "Formatted at 2026-08-11 09:12:53 on dist-test-slave-xt4k"
server_key: "ff3b9f27bcfa828e6aa29abaf4356386"
server_key_iv: "4e48c375a29988a27f37818e7e4c3937"
server_key_version: "encryptionkey@0"
I20260811 09:12:53.036938 30000 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439250192320-29290-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439250192320-29290-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439250192320-29290-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:12:53.053942 30000 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:12:53.054980 30000 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:12:53.055351 30000 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:12:53.056619 30000 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:12:53.058691 30000 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:12:53.058809 30000 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:12:53.058934 30000 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:12:53.059015 30000 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:12:53.100066 30000 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.130:37353
I20260811 09:12:53.100119 30232 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.130:37353 every 8 connection(s)
I20260811 09:12:53.101603 30000 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439250192320-29290-0/minicluster-data/ts-1/data/info.pb
I20260811 09:12:53.106452 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 30000
I20260811 09:12:53.106624 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439250192320-29290-0/minicluster-data/ts-1/wal/instance
I20260811 09:12:53.107092 29290 external_mini_cluster.cc:1428] Setting key d511b50d96d0a8a44088b090de1f49ac
I20260811 09:12:53.110216 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439250192320-29290-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439250192320-29290-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439250192320-29290-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439250192320-29290-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.154.131:0
--local_ip_for_outbound_sockets=127.28.154.131
--webserver_interface=127.28.154.131
--webserver_port=0
--tserver_master_addrs=127.28.154.190:33749
--builtin_ntp_servers=127.28.154.148:39777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 09:12:53.115008 29996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:53.104288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.130:40085 (local address 127.28.154.190:33749)
0811 09:12:53.104455 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0811 09:12:53.104459 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:12:53.105915 (+ 1456us) server_negotiation.cc:407] Connection header received
0811 09:12:53.107780 (+ 1865us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:12:53.107787 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:12:53.107870 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:12:53.107964 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:12:53.109199 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:53.110278 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:53.111981 (+ 1703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:53.112540 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:53.112661 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:12:53.114127 (+ 1466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:12:53.114133 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:12:53.114137 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:12:53.114160 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:12:53.114197 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:12:53.114211 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:12:53.114300 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 09:12:53.114593 (+ 293us) server_negotiation.cc:299] Negotiation successful
0811 09:12:53.114694 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 09:12:53.116146 30239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:53.104866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:33749 (local address 127.28.154.130:40085)
0811 09:12:53.105766 (+ 900us) negotiation.cc:107] Waiting for socket to connect
0811 09:12:53.105801 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0811 09:12:53.107493 (+ 1692us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:12:53.108218 (+ 725us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:12:53.108227 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:12:53.108319 (+ 92us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:12:53.109017 (+ 698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:12:53.109038 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:12:53.110475 (+ 1437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:12:53.110480 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:12:53.111789 (+ 1309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:12:53.111803 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:12:53.112789 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:12:53.112795 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:12:53.113065 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:12:53.113769 (+ 704us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 09:12:53.113772 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:12:53.113816 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 09:12:53.113903 (+ 87us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:12:53.114346 (+ 443us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:12:53.114360 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:12:53.114369 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 09:12:53.114568 (+ 199us) client_negotiation.cc:240] Negotiation successful
0811 09:12:53.115011 (+ 443us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":256,"threads_started":1}
I20260811 09:12:53.117870 30235 heartbeater.cc:347] Connected to a master server at 127.28.154.190:33749
I20260811 09:12:53.118214 30235 heartbeater.cc:464] Registering TS with master...
I20260811 09:12:53.119616 30235 heartbeater.cc:511] Master 127.28.154.190:33749 requested a full tablet report, sending...
I20260811 09:12:53.121997 29635 ts_manager.cc:194] Registered new tserver with Master: c2f51e4313c5403aa360293afbb6b0fd (127.28.154.130:37353)
I20260811 09:12:53.123013 29635 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.28.154.130:40085
W20260811 09:12:53.455572 30242 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:12:53.455879 30242 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:12:53.456007 30242 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:12:53.463464 30242 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:12:53.463585 30242 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:12:53.463728 30242 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.154.131
I20260811 09:12:53.472960 30242 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:39777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439250192320-29290-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439250192320-29290-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.154.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439250192320-29290-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.154.131
--webserver_port=0
--tserver_master_addrs=127.28.154.190:33749
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.154.131
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439250192320-29290-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:12:53.474329 30242 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:12:53.476143 30242 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:12:53.489260 30253 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:12:53.489380 30255 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:12:53.489226 30252 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:12:53.490031 30242 server_base.cc:1034] running on GCE node
I20260811 09:12:53.490706 30242 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:12:53.491786 30242 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:12:53.493013 30242 hybrid_clock.cc:648] HybridClock initialized: now 1786439573492960 us; error 63 us; skew 500 ppm
I20260811 09:12:53.493377 30242 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 09:12:53.495729 30242 webserver.cc:460] Webserver started at http://127.28.154.131:36003/ using document root <none> and password file <none>
I20260811 09:12:53.496505 30242 fs_manager.cc:359] Metadata directory not provided
I20260811 09:12:53.496621 30242 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:12:53.496932 30242 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:12:53.500072 30242 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439250192320-29290-0/minicluster-data/ts-2/data/instance:
uuid: "b6eaf9bd684b41c1821a9e9da3e2c709"
format_stamp: "Formatted at 2026-08-11 09:12:53 on dist-test-slave-xt4k"
server_key: "19c120f6ed0819da17d73d44979a3053"
server_key_iv: "53adec5395009f6a0676e70c12d59500"
server_key_version: "encryptionkey@0"
I20260811 09:12:53.500946 30242 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439250192320-29290-0/minicluster-data/ts-2/wal/instance:
uuid: "b6eaf9bd684b41c1821a9e9da3e2c709"
format_stamp: "Formatted at 2026-08-11 09:12:53 on dist-test-slave-xt4k"
server_key: "19c120f6ed0819da17d73d44979a3053"
server_key_iv: "53adec5395009f6a0676e70c12d59500"
server_key_version: "encryptionkey@0"
I20260811 09:12:53.506420 30242 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 09:12:53.510406 30268 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:12:53.511915 30242 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 09:12:53.512082 30242 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439250192320-29290-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439250192320-29290-0/minicluster-data/ts-2/wal
uuid: "b6eaf9bd684b41c1821a9e9da3e2c709"
format_stamp: "Formatted at 2026-08-11 09:12:53 on dist-test-slave-xt4k"
server_key: "19c120f6ed0819da17d73d44979a3053"
server_key_iv: "53adec5395009f6a0676e70c12d59500"
server_key_version: "encryptionkey@0"
I20260811 09:12:53.512298 30242 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439250192320-29290-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439250192320-29290-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439250192320-29290-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:12:53.563187 30242 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:12:53.564145 30242 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:12:53.564500 30242 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:12:53.565579 30242 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:12:53.567587 30242 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:12:53.567713 30242 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:12:53.567829 30242 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:12:53.567915 30242 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:12:53.600503 30242 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.131:36563
I20260811 09:12:53.600564 30500 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.131:36563 every 8 connection(s)
I20260811 09:12:53.601799 30242 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439250192320-29290-0/minicluster-data/ts-2/data/info.pb
I20260811 09:12:53.603778 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 30242
I20260811 09:12:53.603964 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439250192320-29290-0/minicluster-data/ts-2/wal/instance
I20260811 09:12:53.604411 29290 external_mini_cluster.cc:1428] Setting key 33eb0adcc72233f03dfd176ebdb01a79
I20260811 09:12:53.616580 29996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:53.605227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.131:43529 (local address 127.28.154.190:33749)
0811 09:12:53.605423 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 09:12:53.605429 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:12:53.607123 (+ 1694us) server_negotiation.cc:407] Connection header received
0811 09:12:53.609000 (+ 1877us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:12:53.609004 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:12:53.609078 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:12:53.609168 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:12:53.610269 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:53.611146 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:53.613591 (+ 2445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:53.614027 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:53.614132 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:12:53.615709 (+ 1577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:12:53.615716 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:12:53.615719 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:12:53.615740 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:12:53.615774 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:12:53.615786 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:12:53.615862 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 09:12:53.616162 (+ 300us) server_negotiation.cc:299] Negotiation successful
0811 09:12:53.616266 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 09:12:53.617798 30503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:53.605841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:33749 (local address 127.28.154.131:43529)
0811 09:12:53.606971 (+ 1130us) negotiation.cc:107] Waiting for socket to connect
0811 09:12:53.607023 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 09:12:53.608727 (+ 1704us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:12:53.609365 (+ 638us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:12:53.609373 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:12:53.609495 (+ 122us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:12:53.610094 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:12:53.610111 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:12:53.611322 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:12:53.611328 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:12:53.613403 (+ 2075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:12:53.613419 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:12:53.614263 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:12:53.614271 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:12:53.614567 (+ 296us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:12:53.615323 (+ 756us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 09:12:53.615327 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:12:53.615378 (+ 51us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 09:12:53.615481 (+ 103us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:12:53.615888 (+ 407us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:12:53.615900 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:12:53.615908 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 09:12:53.616110 (+ 202us) client_negotiation.cc:240] Negotiation successful
0811 09:12:53.616597 (+ 487us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":594,"thread_start_us":358,"threads_started":1}
I20260811 09:12:53.619262 30501 heartbeater.cc:347] Connected to a master server at 127.28.154.190:33749
I20260811 09:12:53.619601 30501 heartbeater.cc:464] Registering TS with master...
I20260811 09:12:53.620368 30501 heartbeater.cc:511] Master 127.28.154.190:33749 requested a full tablet report, sending...
I20260811 09:12:53.621721 29635 ts_manager.cc:194] Registered new tserver with Master: b6eaf9bd684b41c1821a9e9da3e2c709 (127.28.154.131:36563)
I20260811 09:12:53.622511 29635 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.28.154.131:43529
I20260811 09:12:53.632020 29290 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 09:12:53.637637 29992 heartbeater.cc:503] Master 127.28.154.190:33749 was elected leader, sending a full tablet report...
I20260811 09:12:53.642630 29996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:53.636255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44130 (local address 127.28.154.190:33749)
0811 09:12:53.636555 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0811 09:12:53.636562 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:12:53.636825 (+ 263us) server_negotiation.cc:407] Connection header received
0811 09:12:53.637005 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:12:53.637009 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:12:53.637086 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:12:53.637170 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:12:53.637717 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:53.638656 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:53.640220 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:53.640740 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:53.640841 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:12:53.641912 (+ 1071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:12:53.641918 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:12:53.641921 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:12:53.641939 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:12:53.641971 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:12:53.641984 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:12:53.642080 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 09:12:53.642220 (+ 140us) server_negotiation.cc:299] Negotiation successful
0811 09:12:53.642318 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138}
I20260811 09:12:53.646584 29635 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44130:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 09:12:53.653573 29635 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 09:12:53.676177 30519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:53.666007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.129:33867 (local address 127.0.0.1:36460)
0811 09:12:53.666810 (+ 803us) negotiation.cc:107] Waiting for socket to connect
0811 09:12:53.666854 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 09:12:53.667028 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:12:53.668512 (+ 1484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:12:53.668520 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:12:53.668550 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:12:53.669039 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:12:53.669053 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:12:53.670874 (+ 1821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:12:53.670880 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:12:53.673493 (+ 2613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:12:53.673510 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:12:53.675160 (+ 1650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:12:53.675166 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:12:53.675379 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:12:53.675440 (+ 61us) client_negotiation.cc:769] Sending connection context
0811 09:12:53.675554 (+ 114us) client_negotiation.cc:240] Negotiation successful
0811 09:12:53.675774 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556,"mutex_wait_us":63,"thread_start_us":369,"threads_started":1}
I20260811 09:12:53.676829 30520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:53.666138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.131:36563 (local address 127.0.0.1:55092)
0811 09:12:53.667587 (+ 1449us) negotiation.cc:107] Waiting for socket to connect
0811 09:12:53.667605 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 09:12:53.667731 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:12:53.668784 (+ 1053us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:12:53.668788 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:12:53.668816 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:12:53.669762 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:12:53.669772 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:12:53.671525 (+ 1753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:12:53.671534 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:12:53.673731 (+ 2197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:12:53.673742 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:12:53.675966 (+ 2224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:12:53.675975 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:12:53.676229 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:12:53.676264 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 09:12:53.676372 (+ 108us) client_negotiation.cc:240] Negotiation successful
0811 09:12:53.676501 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1320,"mutex_wait_us":616,"thread_start_us":1736,"threads_started":1}
I20260811 09:12:53.676836 30521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:53.666257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36460 (local address 127.28.154.129:33867)
0811 09:12:53.667984 (+ 1727us) server_negotiation.cc:206] Beginning negotiation
0811 09:12:53.667991 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:12:53.668014 (+ 23us) server_negotiation.cc:407] Connection header received
0811 09:12:53.668090 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:12:53.668096 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:12:53.668278 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:12:53.669062 (+ 784us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:12:53.669301 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:53.670705 (+ 1404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:53.673747 (+ 3042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:53.674951 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:53.675996 (+ 1045us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:12:53.676193 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0811 09:12:53.676267 (+ 74us) server_negotiation.cc:299] Negotiation successful
0811 09:12:53.676488 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1561,"thread_start_us":1437,"threads_started":1}
I20260811 09:12:53.676993 30522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:53.666767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55092 (local address 127.28.154.131:36563)
0811 09:12:53.667532 (+ 765us) server_negotiation.cc:206] Beginning negotiation
0811 09:12:53.667541 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 09:12:53.667847 (+ 306us) server_negotiation.cc:407] Connection header received
0811 09:12:53.667955 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:12:53.667963 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:12:53.668227 (+ 264us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:12:53.668487 (+ 260us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:12:53.669934 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:53.671346 (+ 1412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:53.673913 (+ 2567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:53.675307 (+ 1394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:53.675525 (+ 218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:12:53.675756 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0811 09:12:53.676458 (+ 702us) server_negotiation.cc:299] Negotiation successful
0811 09:12:53.676645 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":583,"thread_start_us":406,"threads_started":1}
I20260811 09:12:53.677670 30523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:53.667638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.130:37353 (local address 127.0.0.1:59230)
0811 09:12:53.669318 (+ 1680us) negotiation.cc:107] Waiting for socket to connect
0811 09:12:53.669337 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 09:12:53.669462 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:12:53.670263 (+ 801us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:12:53.670268 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:12:53.670295 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:12:53.670652 (+ 357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:12:53.670662 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:12:53.672282 (+ 1620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:12:53.672287 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:12:53.673887 (+ 1600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:12:53.673902 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:12:53.675920 (+ 2018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:12:53.675928 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:12:53.676174 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:12:53.676231 (+ 57us) client_negotiation.cc:769] Sending connection context
0811 09:12:53.677241 (+ 1010us) client_negotiation.cc:240] Negotiation successful
0811 09:12:53.677369 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1555,"mutex_wait_us":35,"thread_start_us":1399,"threads_started":1}
I20260811 09:12:53.678468 30524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:53.668104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59230 (local address 127.28.154.130:37353)
0811 09:12:53.669697 (+ 1593us) server_negotiation.cc:206] Beginning negotiation
0811 09:12:53.669705 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:12:53.669732 (+ 27us) server_negotiation.cc:407] Connection header received
0811 09:12:53.669837 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:12:53.669844 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:12:53.670069 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:12:53.670251 (+ 182us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:12:53.670832 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:53.672149 (+ 1317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:53.674090 (+ 1941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:53.675427 (+ 1337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:53.675569 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:12:53.675756 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0811 09:12:53.677971 (+ 2215us) server_negotiation.cc:299] Negotiation successful
0811 09:12:53.678157 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1435,"thread_start_us":1327,"threads_started":1}
I20260811 09:12:53.680738 29874 tablet_service.cc:1467] Processing CreateTablet for tablet c51cf49db55b45789154d9cf8cd5fda0 (DEFAULT_TABLE table=test-table [id=30524d353cc0418092ed61b122bb7011]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:12:53.680754 30368 tablet_service.cc:1467] Processing CreateTablet for tablet c51cf49db55b45789154d9cf8cd5fda0 (DEFAULT_TABLE table=test-table [id=30524d353cc0418092ed61b122bb7011]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:12:53.682025 29874 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c51cf49db55b45789154d9cf8cd5fda0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:12:53.682190 30368 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c51cf49db55b45789154d9cf8cd5fda0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:12:53.681941 30115 tablet_service.cc:1467] Processing CreateTablet for tablet c51cf49db55b45789154d9cf8cd5fda0 (DEFAULT_TABLE table=test-table [id=30524d353cc0418092ed61b122bb7011]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:12:53.683260 30115 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c51cf49db55b45789154d9cf8cd5fda0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:12:53.696666 30528 tablet_bootstrap.cc:492] T c51cf49db55b45789154d9cf8cd5fda0 P 94f08e6fe92a433c986fb6872f4da8bc: Bootstrap starting.
I20260811 09:12:53.697676 30530 tablet_bootstrap.cc:492] T c51cf49db55b45789154d9cf8cd5fda0 P c2f51e4313c5403aa360293afbb6b0fd: Bootstrap starting.
I20260811 09:12:53.701694 30530 tablet_bootstrap.cc:654] T c51cf49db55b45789154d9cf8cd5fda0 P c2f51e4313c5403aa360293afbb6b0fd: Neither blocks nor log segments found. Creating new log.
I20260811 09:12:53.702050 30528 tablet_bootstrap.cc:654] T c51cf49db55b45789154d9cf8cd5fda0 P 94f08e6fe92a433c986fb6872f4da8bc: Neither blocks nor log segments found. Creating new log.
I20260811 09:12:53.702849 30529 tablet_bootstrap.cc:492] T c51cf49db55b45789154d9cf8cd5fda0 P b6eaf9bd684b41c1821a9e9da3e2c709: Bootstrap starting.
I20260811 09:12:53.703315 30530 log.cc:824] T c51cf49db55b45789154d9cf8cd5fda0 P c2f51e4313c5403aa360293afbb6b0fd: Log is configured to *not* fsync() on all Append() calls
I20260811 09:12:53.703353 30528 log.cc:824] T c51cf49db55b45789154d9cf8cd5fda0 P 94f08e6fe92a433c986fb6872f4da8bc: Log is configured to *not* fsync() on all Append() calls
I20260811 09:12:53.706864 30529 tablet_bootstrap.cc:654] T c51cf49db55b45789154d9cf8cd5fda0 P b6eaf9bd684b41c1821a9e9da3e2c709: Neither blocks nor log segments found. Creating new log.
I20260811 09:12:53.707005 30530 tablet_bootstrap.cc:492] T c51cf49db55b45789154d9cf8cd5fda0 P c2f51e4313c5403aa360293afbb6b0fd: No bootstrap required, opened a new log
I20260811 09:12:53.707005 30528 tablet_bootstrap.cc:492] T c51cf49db55b45789154d9cf8cd5fda0 P 94f08e6fe92a433c986fb6872f4da8bc: No bootstrap required, opened a new log
I20260811 09:12:53.707319 30528 ts_tablet_manager.cc:1397] T c51cf49db55b45789154d9cf8cd5fda0 P 94f08e6fe92a433c986fb6872f4da8bc: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.002s
I20260811 09:12:53.707319 30530 ts_tablet_manager.cc:1397] T c51cf49db55b45789154d9cf8cd5fda0 P c2f51e4313c5403aa360293afbb6b0fd: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.007s
I20260811 09:12:53.708524 30529 log.cc:824] T c51cf49db55b45789154d9cf8cd5fda0 P b6eaf9bd684b41c1821a9e9da3e2c709: Log is configured to *not* fsync() on all Append() calls
I20260811 09:12:53.711239 30529 tablet_bootstrap.cc:492] T c51cf49db55b45789154d9cf8cd5fda0 P b6eaf9bd684b41c1821a9e9da3e2c709: No bootstrap required, opened a new log
I20260811 09:12:53.711514 30529 ts_tablet_manager.cc:1397] T c51cf49db55b45789154d9cf8cd5fda0 P b6eaf9bd684b41c1821a9e9da3e2c709: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260811 09:12:53.718173 30530 raft_consensus.cc:348] T c51cf49db55b45789154d9cf8cd5fda0 P c2f51e4313c5403aa360293afbb6b0fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94f08e6fe92a433c986fb6872f4da8bc" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 33867 } } peers { permanent_uuid: "b6eaf9bd684b41c1821a9e9da3e2c709" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 36563 } } peers { permanent_uuid: "c2f51e4313c5403aa360293afbb6b0fd" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 37353 } }
I20260811 09:12:53.718179 30528 raft_consensus.cc:348] T c51cf49db55b45789154d9cf8cd5fda0 P 94f08e6fe92a433c986fb6872f4da8bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94f08e6fe92a433c986fb6872f4da8bc" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 33867 } } peers { permanent_uuid: "b6eaf9bd684b41c1821a9e9da3e2c709" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 36563 } } peers { permanent_uuid: "c2f51e4313c5403aa360293afbb6b0fd" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 37353 } }
I20260811 09:12:53.718691 30530 raft_consensus.cc:374] T c51cf49db55b45789154d9cf8cd5fda0 P c2f51e4313c5403aa360293afbb6b0fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:12:53.718691 30528 raft_consensus.cc:374] T c51cf49db55b45789154d9cf8cd5fda0 P 94f08e6fe92a433c986fb6872f4da8bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:12:53.718792 30530 raft_consensus.cc:729] T c51cf49db55b45789154d9cf8cd5fda0 P c2f51e4313c5403aa360293afbb6b0fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2f51e4313c5403aa360293afbb6b0fd, State: Initialized, Role: FOLLOWER
I20260811 09:12:53.718789 30528 raft_consensus.cc:729] T c51cf49db55b45789154d9cf8cd5fda0 P 94f08e6fe92a433c986fb6872f4da8bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94f08e6fe92a433c986fb6872f4da8bc, State: Initialized, Role: FOLLOWER
I20260811 09:12:53.719337 30529 raft_consensus.cc:348] T c51cf49db55b45789154d9cf8cd5fda0 P b6eaf9bd684b41c1821a9e9da3e2c709 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94f08e6fe92a433c986fb6872f4da8bc" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 33867 } } peers { permanent_uuid: "b6eaf9bd684b41c1821a9e9da3e2c709" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 36563 } } peers { permanent_uuid: "c2f51e4313c5403aa360293afbb6b0fd" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 37353 } }
I20260811 09:12:53.719456 30528 consensus_queue.cc:260] T c51cf49db55b45789154d9cf8cd5fda0 P 94f08e6fe92a433c986fb6872f4da8bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94f08e6fe92a433c986fb6872f4da8bc" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 33867 } } peers { permanent_uuid: "b6eaf9bd684b41c1821a9e9da3e2c709" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 36563 } } peers { permanent_uuid: "c2f51e4313c5403aa360293afbb6b0fd" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 37353 } }
I20260811 09:12:53.719456 30530 consensus_queue.cc:260] T c51cf49db55b45789154d9cf8cd5fda0 P c2f51e4313c5403aa360293afbb6b0fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94f08e6fe92a433c986fb6872f4da8bc" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 33867 } } peers { permanent_uuid: "b6eaf9bd684b41c1821a9e9da3e2c709" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 36563 } } peers { permanent_uuid: "c2f51e4313c5403aa360293afbb6b0fd" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 37353 } }
I20260811 09:12:53.719776 30529 raft_consensus.cc:374] T c51cf49db55b45789154d9cf8cd5fda0 P b6eaf9bd684b41c1821a9e9da3e2c709 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:12:53.719873 30529 raft_consensus.cc:729] T c51cf49db55b45789154d9cf8cd5fda0 P b6eaf9bd684b41c1821a9e9da3e2c709 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6eaf9bd684b41c1821a9e9da3e2c709, State: Initialized, Role: FOLLOWER
I20260811 09:12:53.720611 30529 consensus_queue.cc:260] T c51cf49db55b45789154d9cf8cd5fda0 P b6eaf9bd684b41c1821a9e9da3e2c709 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94f08e6fe92a433c986fb6872f4da8bc" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 33867 } } peers { permanent_uuid: "b6eaf9bd684b41c1821a9e9da3e2c709" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 36563 } } peers { permanent_uuid: "c2f51e4313c5403aa360293afbb6b0fd" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 37353 } }
I20260811 09:12:53.721583 30235 heartbeater.cc:503] Master 127.28.154.190:33749 was elected leader, sending a full tablet report...
I20260811 09:12:53.721760 30530 ts_tablet_manager.cc:1428] T c51cf49db55b45789154d9cf8cd5fda0 P c2f51e4313c5403aa360293afbb6b0fd: Time spent starting tablet: real 0.014s user 0.008s sys 0.007s
I20260811 09:12:53.722123 30528 ts_tablet_manager.cc:1428] T c51cf49db55b45789154d9cf8cd5fda0 P 94f08e6fe92a433c986fb6872f4da8bc: Time spent starting tablet: real 0.015s user 0.015s sys 0.001s
I20260811 09:12:53.723026 30501 heartbeater.cc:503] Master 127.28.154.190:33749 was elected leader, sending a full tablet report...
I20260811 09:12:53.723609 30529 ts_tablet_manager.cc:1428] T c51cf49db55b45789154d9cf8cd5fda0 P b6eaf9bd684b41c1821a9e9da3e2c709: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
I20260811 09:12:53.752127 30535 raft_consensus.cc:482] T c51cf49db55b45789154d9cf8cd5fda0 P c2f51e4313c5403aa360293afbb6b0fd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:12:53.752547 30535 raft_consensus.cc:504] T c51cf49db55b45789154d9cf8cd5fda0 P c2f51e4313c5403aa360293afbb6b0fd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94f08e6fe92a433c986fb6872f4da8bc" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 33867 } } peers { permanent_uuid: "b6eaf9bd684b41c1821a9e9da3e2c709" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 36563 } } peers { permanent_uuid: "c2f51e4313c5403aa360293afbb6b0fd" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 37353 } }
I20260811 09:12:53.754356 30535 leader_election.cc:288] T c51cf49db55b45789154d9cf8cd5fda0 P c2f51e4313c5403aa360293afbb6b0fd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 94f08e6fe92a433c986fb6872f4da8bc (127.28.154.129:33867), b6eaf9bd684b41c1821a9e9da3e2c709 (127.28.154.131:36563)
I20260811 09:12:53.763796 30522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:53.754911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.130:46055 (local address 127.28.154.131:36563)
0811 09:12:53.755087 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 09:12:53.755093 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:12:53.756160 (+ 1067us) server_negotiation.cc:407] Connection header received
0811 09:12:53.756354 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:12:53.756358 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:12:53.756420 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:12:53.756484 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:12:53.757096 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:53.758127 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:53.761248 (+ 3121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:53.762559 (+ 1311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:53.762746 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:12:53.762822 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 09:12:53.763258 (+ 436us) server_negotiation.cc:299] Negotiation successful
0811 09:12:53.763535 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 09:12:53.763796 30521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:53.754844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.130:60857 (local address 127.28.154.129:33867)
0811 09:12:53.755089 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0811 09:12:53.755095 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:12:53.755766 (+ 671us) server_negotiation.cc:407] Connection header received
0811 09:12:53.755974 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:12:53.755978 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:12:53.756080 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:12:53.756164 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:12:53.756875 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:53.758041 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:53.761247 (+ 3206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:53.762559 (+ 1312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:53.762746 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:12:53.762822 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 09:12:53.763258 (+ 436us) server_negotiation.cc:299] Negotiation successful
0811 09:12:53.763513 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 09:12:53.763808 30540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:53.754878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.129:33867 (local address 127.28.154.130:60857)
0811 09:12:53.755637 (+ 759us) negotiation.cc:107] Waiting for socket to connect
0811 09:12:53.755660 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 09:12:53.755824 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:12:53.756211 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:12:53.756216 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:12:53.756239 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:12:53.756648 (+ 409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:12:53.756663 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:12:53.758194 (+ 1531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:12:53.758200 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:12:53.760961 (+ 2761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:12:53.760982 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:12:53.762841 (+ 1859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:12:53.762848 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:12:53.763033 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:12:53.763076 (+ 43us) client_negotiation.cc:769] Sending connection context
0811 09:12:53.763181 (+ 105us) client_negotiation.cc:240] Negotiation successful
0811 09:12:53.763351 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":612,"mutex_wait_us":174,"spinlock_wait_cycles":59776,"thread_start_us":426,"threads_started":1}
I20260811 09:12:53.763830 30541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:53.754876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.131:36563 (local address 127.28.154.130:46055)
0811 09:12:53.756047 (+ 1171us) negotiation.cc:107] Waiting for socket to connect
0811 09:12:53.756062 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 09:12:53.756189 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:12:53.756543 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:12:53.756548 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:12:53.756578 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:12:53.756929 (+ 351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:12:53.756938 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:12:53.758259 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:12:53.758263 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:12:53.760952 (+ 2689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:12:53.760975 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:12:53.762816 (+ 1841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:12:53.762827 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:12:53.763019 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:12:53.763084 (+ 65us) client_negotiation.cc:769] Sending connection context
0811 09:12:53.763181 (+ 97us) client_negotiation.cc:240] Negotiation successful
0811 09:12:53.763351 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1065,"mutex_wait_us":209,"thread_start_us":815,"threads_started":1}
I20260811 09:12:53.765218 30415 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c51cf49db55b45789154d9cf8cd5fda0" candidate_uuid: "c2f51e4313c5403aa360293afbb6b0fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6eaf9bd684b41c1821a9e9da3e2c709" is_pre_election: true
I20260811 09:12:53.765830 30415 raft_consensus.cc:2454] T c51cf49db55b45789154d9cf8cd5fda0 P b6eaf9bd684b41c1821a9e9da3e2c709 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2f51e4313c5403aa360293afbb6b0fd in term 0.
I20260811 09:12:53.766127 29911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c51cf49db55b45789154d9cf8cd5fda0" candidate_uuid: "c2f51e4313c5403aa360293afbb6b0fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94f08e6fe92a433c986fb6872f4da8bc" is_pre_election: true
I20260811 09:12:53.766688 29911 raft_consensus.cc:2454] T c51cf49db55b45789154d9cf8cd5fda0 P 94f08e6fe92a433c986fb6872f4da8bc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c2f51e4313c5403aa360293afbb6b0fd in term 0.
I20260811 09:12:53.766737 30027 leader_election.cc:302] T c51cf49db55b45789154d9cf8cd5fda0 P c2f51e4313c5403aa360293afbb6b0fd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b6eaf9bd684b41c1821a9e9da3e2c709, c2f51e4313c5403aa360293afbb6b0fd; no voters:
I20260811 09:12:53.767208 30535 raft_consensus.cc:2781] T c51cf49db55b45789154d9cf8cd5fda0 P c2f51e4313c5403aa360293afbb6b0fd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 09:12:53.767365 30535 raft_consensus.cc:482] T c51cf49db55b45789154d9cf8cd5fda0 P c2f51e4313c5403aa360293afbb6b0fd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 09:12:53.767482 30535 raft_consensus.cc:3037] T c51cf49db55b45789154d9cf8cd5fda0 P c2f51e4313c5403aa360293afbb6b0fd [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:12:53.770154 30535 raft_consensus.cc:504] T c51cf49db55b45789154d9cf8cd5fda0 P c2f51e4313c5403aa360293afbb6b0fd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94f08e6fe92a433c986fb6872f4da8bc" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 33867 } } peers { permanent_uuid: "b6eaf9bd684b41c1821a9e9da3e2c709" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 36563 } } peers { permanent_uuid: "c2f51e4313c5403aa360293afbb6b0fd" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 37353 } }
I20260811 09:12:53.770876 30535 leader_election.cc:288] T c51cf49db55b45789154d9cf8cd5fda0 P c2f51e4313c5403aa360293afbb6b0fd [CANDIDATE]: Term 1 election: Requested vote from peers 94f08e6fe92a433c986fb6872f4da8bc (127.28.154.129:33867), b6eaf9bd684b41c1821a9e9da3e2c709 (127.28.154.131:36563)
I20260811 09:12:53.771534 29911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c51cf49db55b45789154d9cf8cd5fda0" candidate_uuid: "c2f51e4313c5403aa360293afbb6b0fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94f08e6fe92a433c986fb6872f4da8bc"
I20260811 09:12:53.771559 30415 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c51cf49db55b45789154d9cf8cd5fda0" candidate_uuid: "c2f51e4313c5403aa360293afbb6b0fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6eaf9bd684b41c1821a9e9da3e2c709"
I20260811 09:12:53.771780 30415 raft_consensus.cc:3037] T c51cf49db55b45789154d9cf8cd5fda0 P b6eaf9bd684b41c1821a9e9da3e2c709 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:12:53.771778 29911 raft_consensus.cc:3037] T c51cf49db55b45789154d9cf8cd5fda0 P 94f08e6fe92a433c986fb6872f4da8bc [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:12:53.775141 30415 raft_consensus.cc:2454] T c51cf49db55b45789154d9cf8cd5fda0 P b6eaf9bd684b41c1821a9e9da3e2c709 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2f51e4313c5403aa360293afbb6b0fd in term 1.
I20260811 09:12:53.775134 29911 raft_consensus.cc:2454] T c51cf49db55b45789154d9cf8cd5fda0 P 94f08e6fe92a433c986fb6872f4da8bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c2f51e4313c5403aa360293afbb6b0fd in term 1.
I20260811 09:12:53.775667 30027 leader_election.cc:302] T c51cf49db55b45789154d9cf8cd5fda0 P c2f51e4313c5403aa360293afbb6b0fd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 94f08e6fe92a433c986fb6872f4da8bc, b6eaf9bd684b41c1821a9e9da3e2c709, c2f51e4313c5403aa360293afbb6b0fd; no voters:
I20260811 09:12:53.776127 30535 raft_consensus.cc:2781] T c51cf49db55b45789154d9cf8cd5fda0 P c2f51e4313c5403aa360293afbb6b0fd [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:12:53.776656 30535 raft_consensus.cc:686] T c51cf49db55b45789154d9cf8cd5fda0 P c2f51e4313c5403aa360293afbb6b0fd [term 1 LEADER]: Becoming Leader. State: Replica: c2f51e4313c5403aa360293afbb6b0fd, State: Running, Role: LEADER
I20260811 09:12:53.777238 30535 consensus_queue.cc:237] T c51cf49db55b45789154d9cf8cd5fda0 P c2f51e4313c5403aa360293afbb6b0fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94f08e6fe92a433c986fb6872f4da8bc" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 33867 } } peers { permanent_uuid: "b6eaf9bd684b41c1821a9e9da3e2c709" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 36563 } } peers { permanent_uuid: "c2f51e4313c5403aa360293afbb6b0fd" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 37353 } }
I20260811 09:12:53.782322 29635 catalog_manager.cc:5495] T c51cf49db55b45789154d9cf8cd5fda0 P c2f51e4313c5403aa360293afbb6b0fd reported cstate change: term changed from 0 to 1, leader changed from <none> to c2f51e4313c5403aa360293afbb6b0fd (127.28.154.130). New cstate: current_term: 1 leader_uuid: "c2f51e4313c5403aa360293afbb6b0fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94f08e6fe92a433c986fb6872f4da8bc" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 33867 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b6eaf9bd684b41c1821a9e9da3e2c709" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 36563 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c2f51e4313c5403aa360293afbb6b0fd" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 37353 } health_report { overall_health: HEALTHY } } }
I20260811 09:12:53.806562 30524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:53.801080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59244 (local address 127.28.154.130:37353)
0811 09:12:53.801302 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0811 09:12:53.801308 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:12:53.801328 (+ 20us) server_negotiation.cc:407] Connection header received
0811 09:12:53.801557 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:12:53.801570 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:12:53.801699 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:12:53.801838 (+ 139us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:12:53.802217 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:53.802955 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:53.804388 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:53.804821 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:53.804933 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:12:53.805460 (+ 527us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:12:53.805972 (+ 512us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:12:53.806122 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0811 09:12:53.806210 (+ 88us) server_negotiation.cc:299] Negotiation successful
0811 09:12:53.806295 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260811 09:12:53.814963 29911 raft_consensus.cc:1261] T c51cf49db55b45789154d9cf8cd5fda0 P 94f08e6fe92a433c986fb6872f4da8bc [term 1 FOLLOWER]: Refusing update from remote peer c2f51e4313c5403aa360293afbb6b0fd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:12:53.815053 30415 raft_consensus.cc:1261] T c51cf49db55b45789154d9cf8cd5fda0 P b6eaf9bd684b41c1821a9e9da3e2c709 [term 1 FOLLOWER]: Refusing update from remote peer c2f51e4313c5403aa360293afbb6b0fd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:12:53.815999 30542 consensus_queue.cc:1035] T c51cf49db55b45789154d9cf8cd5fda0 P c2f51e4313c5403aa360293afbb6b0fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "94f08e6fe92a433c986fb6872f4da8bc" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 33867 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:12:53.816367 30535 consensus_queue.cc:1035] T c51cf49db55b45789154d9cf8cd5fda0 P c2f51e4313c5403aa360293afbb6b0fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6eaf9bd684b41c1821a9e9da3e2c709" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 36563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:12:53.847370 29635 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44130:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:12:53.847668 29635 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44130:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:12:53.851505 29635 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b435685007cc4b67a6edef334c0ba915: Sending DeleteTablet for 3 replicas of tablet c51cf49db55b45789154d9cf8cd5fda0
I20260811 09:12:53.853119 30115 tablet_service.cc:1514] Processing DeleteTablet for tablet c51cf49db55b45789154d9cf8cd5fda0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:12:53 UTC) from {username='slave'} at 127.0.0.1:59230
I20260811 09:12:53.853119 30368 tablet_service.cc:1514] Processing DeleteTablet for tablet c51cf49db55b45789154d9cf8cd5fda0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:12:53 UTC) from {username='slave'} at 127.0.0.1:55092
I20260811 09:12:53.853230 29874 tablet_service.cc:1514] Processing DeleteTablet for tablet c51cf49db55b45789154d9cf8cd5fda0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:12:53 UTC) from {username='slave'} at 127.0.0.1:36460
I20260811 09:12:53.860739 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 29760
I20260811 09:12:53.866700 30571 tablet_replica.cc:330] T c51cf49db55b45789154d9cf8cd5fda0 P 94f08e6fe92a433c986fb6872f4da8bc: stopping tablet replica
I20260811 09:12:53.867195 30571 raft_consensus.cc:2229] T c51cf49db55b45789154d9cf8cd5fda0 P 94f08e6fe92a433c986fb6872f4da8bc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:12:53.867765 30571 raft_consensus.cc:2258] T c51cf49db55b45789154d9cf8cd5fda0 P 94f08e6fe92a433c986fb6872f4da8bc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:12:53.874241 30570 tablet_replica.cc:330] T c51cf49db55b45789154d9cf8cd5fda0 P c2f51e4313c5403aa360293afbb6b0fd: stopping tablet replica
I20260811 09:12:53.874855 30570 raft_consensus.cc:2229] T c51cf49db55b45789154d9cf8cd5fda0 P c2f51e4313c5403aa360293afbb6b0fd [term 1 LEADER]: Raft consensus shutting down.
I20260811 09:12:53.875759 30570 raft_consensus.cc:2258] T c51cf49db55b45789154d9cf8cd5fda0 P c2f51e4313c5403aa360293afbb6b0fd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:12:53.878355 30571 ts_tablet_manager.cc:1905] T c51cf49db55b45789154d9cf8cd5fda0 P 94f08e6fe92a433c986fb6872f4da8bc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:12:53.879144 30570 ts_tablet_manager.cc:1905] T c51cf49db55b45789154d9cf8cd5fda0 P c2f51e4313c5403aa360293afbb6b0fd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:12:53.880409 30521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:53.862880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36474 (local address 127.28.154.129:33867)
0811 09:12:53.863115 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0811 09:12:53.863122 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:12:53.871150 (+ 8028us) server_negotiation.cc:407] Connection header received
0811 09:12:53.871345 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:12:53.871352 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:12:53.871497 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:12:53.871624 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:12:53.872357 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:53.873595 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:53.875393 (+ 1798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:53.875830 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:53.875953 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:12:53.877909 (+ 1956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:12:53.877933 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:12:53.877953 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:12:53.879313 (+ 1360us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:12:53.879397 (+ 84us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:12:53.879429 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:12:53.879540 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 09:12:53.879847 (+ 307us) server_negotiation.cc:299] Negotiation successful
0811 09:12:53.880010 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260811 09:12:53.881575 29983 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:12:53.881799 30567 tablet_replica.cc:330] T c51cf49db55b45789154d9cf8cd5fda0 P b6eaf9bd684b41c1821a9e9da3e2c709: stopping tablet replica
I20260811 09:12:53.882294 30567 raft_consensus.cc:2229] T c51cf49db55b45789154d9cf8cd5fda0 P b6eaf9bd684b41c1821a9e9da3e2c709 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:12:53.883286 30567 raft_consensus.cc:2258] T c51cf49db55b45789154d9cf8cd5fda0 P b6eaf9bd684b41c1821a9e9da3e2c709 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:12:53.886713 30567 ts_tablet_manager.cc:1905] T c51cf49db55b45789154d9cf8cd5fda0 P b6eaf9bd684b41c1821a9e9da3e2c709: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:12:53.891026 30570 ts_tablet_manager.cc:1918] T c51cf49db55b45789154d9cf8cd5fda0 P c2f51e4313c5403aa360293afbb6b0fd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:12:53.891222 30570 log.cc:1196] T c51cf49db55b45789154d9cf8cd5fda0 P c2f51e4313c5403aa360293afbb6b0fd: Deleting WAL directory at /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439250192320-29290-0/minicluster-data/ts-1/wal/wals/c51cf49db55b45789154d9cf8cd5fda0
I20260811 09:12:53.891782 30570 ts_tablet_manager.cc:1939] T c51cf49db55b45789154d9cf8cd5fda0 P c2f51e4313c5403aa360293afbb6b0fd: Deleting consensus metadata
I20260811 09:12:53.893855 29611 catalog_manager.cc:4841] TS c2f51e4313c5403aa360293afbb6b0fd (127.28.154.130:37353): tablet c51cf49db55b45789154d9cf8cd5fda0 (table test-table [id=30524d353cc0418092ed61b122bb7011]) successfully deleted
I20260811 09:12:53.894027 30567 ts_tablet_manager.cc:1918] T c51cf49db55b45789154d9cf8cd5fda0 P b6eaf9bd684b41c1821a9e9da3e2c709: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:12:53.894214 30567 log.cc:1196] T c51cf49db55b45789154d9cf8cd5fda0 P b6eaf9bd684b41c1821a9e9da3e2c709: Deleting WAL directory at /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786439250192320-29290-0/minicluster-data/ts-2/wal/wals/c51cf49db55b45789154d9cf8cd5fda0
I20260811 09:12:53.895073 30567 ts_tablet_manager.cc:1939] T c51cf49db55b45789154d9cf8cd5fda0 P b6eaf9bd684b41c1821a9e9da3e2c709: Deleting consensus metadata
I20260811 09:12:53.897188 29615 catalog_manager.cc:4841] TS b6eaf9bd684b41c1821a9e9da3e2c709 (127.28.154.131:36563): tablet c51cf49db55b45789154d9cf8cd5fda0 (table test-table [id=30524d353cc0418092ed61b122bb7011]) successfully deleted
I20260811 09:12:53.980489 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 29760
W20260811 09:12:54.014621 29611 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.28.154.129:33867 (error 108)
I20260811 09:12:54.014911 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 30000
I20260811 09:12:54.016916 30523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:54.016395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.129:33867 (local address 127.0.0.1:36480)
0811 09:12:54.016571 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0811 09:12:54.016708 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.129:33867: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260811 09:12:54.017208 29611 catalog_manager.cc:4584] TS 94f08e6fe92a433c986fb6872f4da8bc (127.28.154.129:33867): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c51cf49db55b45789154d9cf8cd5fda0: Network error: Client connection negotiation failed: client connection to 127.28.154.129:33867: connect: Connection refused (error 111)
I20260811 09:12:54.023769 30524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:54.016436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59250 (local address 127.28.154.130:37353)
0811 09:12:54.016609 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 09:12:54.016614 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:12:54.016632 (+ 18us) server_negotiation.cc:407] Connection header received
0811 09:12:54.016699 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:12:54.016704 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:12:54.016801 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:12:54.016912 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:12:54.017551 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:54.018345 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:54.020211 (+ 1866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:54.020900 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:54.021027 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:12:54.022292 (+ 1265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:12:54.022302 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:12:54.022319 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:12:54.022964 (+ 645us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:12:54.023026 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:12:54.023050 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:12:54.023143 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 09:12:54.023306 (+ 163us) server_negotiation.cc:299] Negotiation successful
0811 09:12:54.023397 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 09:12:54.024521 30226 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:12:54.071247 30523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:54.070649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.129:33867 (local address 127.0.0.1:36496)
0811 09:12:54.070922 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0811 09:12:54.071030 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.129:33867: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260811 09:12:54.073168 30523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:54.072686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.129:33867 (local address 127.0.0.1:36504)
0811 09:12:54.072895 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0811 09:12:54.072992 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.129:33867: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260811 09:12:54.133708 30523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:54.133194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.129:33867 (local address 127.0.0.1:36508)
0811 09:12:54.133404 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0811 09:12:54.133500 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.129:33867: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260811 09:12:54.134773 30523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:54.134427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.129:33867 (local address 127.0.0.1:36524)
0811 09:12:54.134583 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0811 09:12:54.134658 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.129:33867: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260811 09:12:54.137385 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 30000
I20260811 09:12:54.163348 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 30242
I20260811 09:12:54.169871 30522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:54.164050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55096 (local address 127.28.154.131:36563)
0811 09:12:54.164210 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0811 09:12:54.164215 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:12:54.164271 (+ 56us) server_negotiation.cc:407] Connection header received
0811 09:12:54.164460 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:12:54.164464 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:12:54.164550 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:12:54.164636 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:12:54.165028 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:54.165770 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:54.167041 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:54.167430 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:54.167520 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:12:54.168639 (+ 1119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:12:54.168649 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:12:54.168664 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:12:54.169235 (+ 571us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:12:54.169285 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:12:54.169305 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:12:54.169388 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 09:12:54.169519 (+ 131us) server_negotiation.cc:299] Negotiation successful
0811 09:12:54.169618 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 09:12:54.170585 30492 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:12:54.215732 30523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:54.215154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.129:33867 (local address 127.0.0.1:36528)
0811 09:12:54.215381 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0811 09:12:54.215554 (+ 173us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.129:33867: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260811 09:12:54.217146 30523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:54.216675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.129:33867 (local address 127.0.0.1:36536)
0811 09:12:54.216907 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0811 09:12:54.217013 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.129:33867: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111}
I20260811 09:12:54.260891 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 30242
I20260811 09:12:54.283066 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 29584
I20260811 09:12:54.284374 29698 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:12:54.395396 30523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:54.394926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.129:33867 (local address 127.0.0.1:36550)
0811 09:12:54.395111 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0811 09:12:54.395223 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.129:33867: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260811 09:12:54.395411 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 29584
2026-08-11T09:12:54Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (2991 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439250192320-29290-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:12:54 dist-test-slave-xt4k krb5kdc[30588](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:12:54 dist-test-slave-xt4k krb5kdc[30588](info): set up 2 sockets
Aug 11 09:12:54 dist-test-slave-xt4k krb5kdc[30588](info): commencing operation
krb5kdc: starting...
W20260811 09:12:55.699085 29290 mini_kdc.cc:121] Time spent starting KDC: real 1.253s user 0.003s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439250192320-29290-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439250192320-29290-0/krb5kdc/test-user.keytab.
Aug 11 09:12:55 dist-test-slave-xt4k krb5kdc[30588](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439575, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:12:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:12:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.154.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.154.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.154.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab.
Entry for principal kudu/127.28.154.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab.
I20260811 09:12:56.016446 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439250192320-29290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439250192320-29290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439250192320-29290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.154.190:36913
--webserver_interface=127.28.154.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab
--principal=kudu/127.28.154.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:44533
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.154.190:36913
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:12:56.366350 30624 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:12:56.366683 30624 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:12:56.366820 30624 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:12:56.374236 30624 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:12:56.374373 30624 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:12:56.374462 30624 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:12:56.374540 30624 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:12:56.374601 30624 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:12:56.384721 30624 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:44533
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439250192320-29290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439250192320-29290-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.154.190:36913
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab
--principal=kudu/127.28.154.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.154.190:36913
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.154.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439250192320-29290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:12:56.386090 30624 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:12:56.387413 30624 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:12:56.399132 30634 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:12:56.399194 30638 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:12:56.399196 30636 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:12:56.400311 30624 server_base.cc:1034] running on GCE node
I20260811 09:12:56.401105 30624 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:12:56.402585 30624 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:12:56.403821 30624 hybrid_clock.cc:648] HybridClock initialized: now 1786439576403770 us; error 39 us; skew 500 ppm
I20260811 09:12:56.404196 30624 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:12:56 dist-test-slave-xt4k krb5kdc[30588](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439576, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:12:56.407555 30624 init.cc:377] Logged in from keytab as kudu/127.28.154.190@KRBTEST.COM (short username kudu)
I20260811 09:12:56.409215 30624 webserver.cc:460] Webserver started at http://127.28.154.190:34721/ using document root <none> and password file <none>
I20260811 09:12:56.409936 30624 fs_manager.cc:359] Metadata directory not provided
I20260811 09:12:56.410063 30624 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:12:56.410382 30624 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:12:56.413519 30624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439250192320-29290-0/minicluster-data/master-0/data/instance:
uuid: "ddd5f0813ca54f1388e68eed82828e48"
format_stamp: "Formatted at 2026-08-11 09:12:56 on dist-test-slave-xt4k"
server_key: "91dc091c4880e11b6ec72d0e5eb49226"
server_key_iv: "927aea7e8674a4193b53a16314775a2d"
server_key_version: "encryptionkey@0"
I20260811 09:12:56.414286 30624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439250192320-29290-0/minicluster-data/master-0/wal/instance:
uuid: "ddd5f0813ca54f1388e68eed82828e48"
format_stamp: "Formatted at 2026-08-11 09:12:56 on dist-test-slave-xt4k"
server_key: "91dc091c4880e11b6ec72d0e5eb49226"
server_key_iv: "927aea7e8674a4193b53a16314775a2d"
server_key_version: "encryptionkey@0"
I20260811 09:12:56.419549 30624 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 09:12:56.423367 30656 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:12:56.424801 30624 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 09:12:56.424996 30624 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439250192320-29290-0/minicluster-data/master-0/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439250192320-29290-0/minicluster-data/master-0/wal
uuid: "ddd5f0813ca54f1388e68eed82828e48"
format_stamp: "Formatted at 2026-08-11 09:12:56 on dist-test-slave-xt4k"
server_key: "91dc091c4880e11b6ec72d0e5eb49226"
server_key_iv: "927aea7e8674a4193b53a16314775a2d"
server_key_version: "encryptionkey@0"
I20260811 09:12:56.425236 30624 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439250192320-29290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439250192320-29290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439250192320-29290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:12:56.449010 30624 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:12:56.453548 30624 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:12:56.453835 30624 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:12:56.472662 30624 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.190:36913
I20260811 09:12:56.472684 30743 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.190:36913 every 8 connection(s)
I20260811 09:12:56.474287 30624 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
I20260811 09:12:56.478288 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 30624
I20260811 09:12:56.478490 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439250192320-29290-0/minicluster-data/master-0/wal/instance
I20260811 09:12:56.478504 30744 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:12:56.479105 29290 external_mini_cluster.cc:1428] Setting key bbf6233662aacb3144ed0724749eb80c
I20260811 09:12:56.493942 30744 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ddd5f0813ca54f1388e68eed82828e48: Bootstrap starting.
Aug 11 09:12:56 dist-test-slave-xt4k krb5kdc[30588](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439575, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.154.190@KRBTEST.COM
I20260811 09:12:56.497611 30744 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ddd5f0813ca54f1388e68eed82828e48: Neither blocks nor log segments found. Creating new log.
I20260811 09:12:56.498723 30744 log.cc:824] T 00000000000000000000000000000000 P ddd5f0813ca54f1388e68eed82828e48: Log is configured to *not* fsync() on all Append() calls
I20260811 09:12:56.501720 30744 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ddd5f0813ca54f1388e68eed82828e48: No bootstrap required, opened a new log
I20260811 09:12:56.503387 30751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:56.481089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54468 (local address 127.28.154.190:36913)
0811 09:12:56.481956 (+ 867us) server_negotiation.cc:206] Beginning negotiation
0811 09:12:56.481969 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 09:12:56.482016 (+ 47us) server_negotiation.cc:407] Connection header received
0811 09:12:56.483594 (+ 1578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:12:56.483617 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:12:56.484039 (+ 422us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:12:56.484420 (+ 381us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:12:56.486447 (+ 2027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:56.487797 (+ 1350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:56.489724 (+ 1927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:56.490511 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:56.490662 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:12:56.495762 (+ 5100us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:12:56.495772 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:12:56.495788 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:12:56.495832 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:12:56.499082 (+ 3250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:12:56.499850 (+ 768us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:12:56.499857 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:12:56.499865 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:12:56.500065 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:12:56.500537 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:12:56.500541 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:12:56.500543 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:12:56.501111 (+ 568us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:12:56.501405 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0811 09:12:56.501922 (+ 517us) server_negotiation.cc:299] Negotiation successful
0811 09:12:56.502398 (+ 476us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":636,"thread_start_us":373,"threads_started":1}
I20260811 09:12:56.509799 30744 raft_consensus.cc:348] T 00000000000000000000000000000000 P ddd5f0813ca54f1388e68eed82828e48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddd5f0813ca54f1388e68eed82828e48" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 36913 } }
I20260811 09:12:56.510217 30744 raft_consensus.cc:374] T 00000000000000000000000000000000 P ddd5f0813ca54f1388e68eed82828e48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:12:56.510311 30744 raft_consensus.cc:729] T 00000000000000000000000000000000 P ddd5f0813ca54f1388e68eed82828e48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ddd5f0813ca54f1388e68eed82828e48, State: Initialized, Role: FOLLOWER
I20260811 09:12:56.510924 30744 consensus_queue.cc:260] T 00000000000000000000000000000000 P ddd5f0813ca54f1388e68eed82828e48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddd5f0813ca54f1388e68eed82828e48" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 36913 } }
I20260811 09:12:56.511188 30744 raft_consensus.cc:388] T 00000000000000000000000000000000 P ddd5f0813ca54f1388e68eed82828e48 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:12:56.511317 30744 raft_consensus.cc:482] T 00000000000000000000000000000000 P ddd5f0813ca54f1388e68eed82828e48 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:12:56.511482 30744 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ddd5f0813ca54f1388e68eed82828e48 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:12:56.514448 30744 raft_consensus.cc:504] T 00000000000000000000000000000000 P ddd5f0813ca54f1388e68eed82828e48 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddd5f0813ca54f1388e68eed82828e48" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 36913 } }
I20260811 09:12:56.515004 30744 leader_election.cc:302] T 00000000000000000000000000000000 P ddd5f0813ca54f1388e68eed82828e48 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ddd5f0813ca54f1388e68eed82828e48; no voters:
I20260811 09:12:56.515612 30744 leader_election.cc:288] T 00000000000000000000000000000000 P ddd5f0813ca54f1388e68eed82828e48 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:12:56.515775 30754 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ddd5f0813ca54f1388e68eed82828e48 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:12:56.516500 30754 raft_consensus.cc:686] T 00000000000000000000000000000000 P ddd5f0813ca54f1388e68eed82828e48 [term 1 LEADER]: Becoming Leader. State: Replica: ddd5f0813ca54f1388e68eed82828e48, State: Running, Role: LEADER
I20260811 09:12:56.517074 30754 consensus_queue.cc:237] T 00000000000000000000000000000000 P ddd5f0813ca54f1388e68eed82828e48 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddd5f0813ca54f1388e68eed82828e48" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 36913 } }
I20260811 09:12:56.517746 30744 sys_catalog.cc:564] T 00000000000000000000000000000000 P ddd5f0813ca54f1388e68eed82828e48 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:12:56.521538 30757 sys_catalog.cc:455] T 00000000000000000000000000000000 P ddd5f0813ca54f1388e68eed82828e48 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ddd5f0813ca54f1388e68eed82828e48. Latest consensus state: current_term: 1 leader_uuid: "ddd5f0813ca54f1388e68eed82828e48" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddd5f0813ca54f1388e68eed82828e48" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 36913 } } }
I20260811 09:12:56.521575 30756 sys_catalog.cc:455] T 00000000000000000000000000000000 P ddd5f0813ca54f1388e68eed82828e48 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ddd5f0813ca54f1388e68eed82828e48" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddd5f0813ca54f1388e68eed82828e48" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 36913 } } }
I20260811 09:12:56.522080 30756 sys_catalog.cc:458] T 00000000000000000000000000000000 P ddd5f0813ca54f1388e68eed82828e48 [sys.catalog]: This master's current role is: LEADER
I20260811 09:12:56.524963 30757 sys_catalog.cc:458] T 00000000000000000000000000000000 P ddd5f0813ca54f1388e68eed82828e48 [sys.catalog]: This master's current role is: LEADER
I20260811 09:12:56.525492 30773 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:12:56.530932 30773 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:12:56.542114 30773 catalog_manager.cc:1347] Generated new cluster ID: e79b15e0d75f4dc09d129dc9130ba594
I20260811 09:12:56.542253 30773 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:12:56.560927 30773 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:12:56.562904 30773 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:12:56.574713 30773 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ddd5f0813ca54f1388e68eed82828e48: Generated new TSK 0
I20260811 09:12:56.575903 30773 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.28.154.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.154.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.154.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.129.keytab.
Entry for principal kudu/127.28.154.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.129.keytab.
I20260811 09:12:56.655886 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439250192320-29290-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439250192320-29290-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439250192320-29290-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439250192320-29290-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.154.129:0
--local_ip_for_outbound_sockets=127.28.154.129
--webserver_interface=127.28.154.129
--webserver_port=0
--tserver_master_addrs=127.28.154.190:36913
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.129.keytab
--principal=kudu/127.28.154.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:44533
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:12:57.019655 30799 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:12:57.019989 30799 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:12:57.020129 30799 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:12:57.027776 30799 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:12:57.027908 30799 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:12:57.028075 30799 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.154.129
I20260811 09:12:57.038307 30799 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:44533
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439250192320-29290-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439250192320-29290-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.129.keytab
--principal=kudu/127.28.154.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.154.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439250192320-29290-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.154.129
--webserver_port=0
--tserver_master_addrs=127.28.154.190:36913
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.154.129
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439250192320-29290-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:12:57.039626 30799 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:12:57.041004 30799 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:12:57.054113 30810 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:12:57.054168 30816 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:12:57.054082 30812 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:12:57.054927 30799 server_base.cc:1034] running on GCE node
I20260811 09:12:57.055599 30799 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:12:57.056571 30799 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:12:57.057814 30799 hybrid_clock.cc:648] HybridClock initialized: now 1786439577057760 us; error 41 us; skew 500 ppm
I20260811 09:12:57.058173 30799 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:12:57 dist-test-slave-xt4k krb5kdc[30588](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439577, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:12:57.065809 30799 init.cc:377] Logged in from keytab as kudu/127.28.154.129@KRBTEST.COM (short username kudu)
I20260811 09:12:57.067401 30799 webserver.cc:460] Webserver started at http://127.28.154.129:36221/ using document root <none> and password file <none>
I20260811 09:12:57.068114 30799 fs_manager.cc:359] Metadata directory not provided
I20260811 09:12:57.068239 30799 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:12:57.068602 30799 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:12:57.071597 30799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439250192320-29290-0/minicluster-data/ts-0/data/instance:
uuid: "6b27f3f959b04fee804d7c32e03866d2"
format_stamp: "Formatted at 2026-08-11 09:12:57 on dist-test-slave-xt4k"
server_key: "6b19ef13c5ec63491755d88d854283d3"
server_key_iv: "2c7c0eaf1e88a7a32d95e44332f14e54"
server_key_version: "encryptionkey@0"
I20260811 09:12:57.072429 30799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439250192320-29290-0/minicluster-data/ts-0/wal/instance:
uuid: "6b27f3f959b04fee804d7c32e03866d2"
format_stamp: "Formatted at 2026-08-11 09:12:57 on dist-test-slave-xt4k"
server_key: "6b19ef13c5ec63491755d88d854283d3"
server_key_iv: "2c7c0eaf1e88a7a32d95e44332f14e54"
server_key_version: "encryptionkey@0"
I20260811 09:12:57.077762 30799 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260811 09:12:57.081621 30832 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:12:57.084004 30799 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260811 09:12:57.084151 30799 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439250192320-29290-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439250192320-29290-0/minicluster-data/ts-0/wal
uuid: "6b27f3f959b04fee804d7c32e03866d2"
format_stamp: "Formatted at 2026-08-11 09:12:57 on dist-test-slave-xt4k"
server_key: "6b19ef13c5ec63491755d88d854283d3"
server_key_iv: "2c7c0eaf1e88a7a32d95e44332f14e54"
server_key_version: "encryptionkey@0"
I20260811 09:12:57.084328 30799 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439250192320-29290-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439250192320-29290-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439250192320-29290-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:12:57.105926 30799 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:12:57.110277 30799 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:12:57.110538 30799 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:12:57.111584 30799 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:12:57.112984 30799 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:12:57.113063 30799 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:12:57.113148 30799 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:12:57.113191 30799 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:12:57.142485 30799 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.129:46333
I20260811 09:12:57.142525 31043 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.129:46333 every 8 connection(s)
I20260811 09:12:57.143754 30799 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439250192320-29290-0/minicluster-data/ts-0/data/info.pb
I20260811 09:12:57.149350 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 30799
I20260811 09:12:57.149520 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439250192320-29290-0/minicluster-data/ts-0/wal/instance
I20260811 09:12:57.149961 29290 external_mini_cluster.cc:1428] Setting key 4133c539efc649633d7ff2a7af68a9f9
Aug 11 09:12:57 dist-test-slave-xt4k krb5kdc[30588](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439577, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.129@KRBTEST.COM for kudu/127.28.154.190@KRBTEST.COM
I20260811 09:12:57.164523 31047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:57.146452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.129:49855 (local address 127.28.154.190:36913)
0811 09:12:57.146916 (+ 464us) server_negotiation.cc:206] Beginning negotiation
0811 09:12:57.146921 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:12:57.148057 (+ 1136us) server_negotiation.cc:407] Connection header received
0811 09:12:57.150105 (+ 2048us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:12:57.150111 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:12:57.150203 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:12:57.150302 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:12:57.152090 (+ 1788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:57.152860 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:57.154435 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:57.154837 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:57.154957 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:12:57.158741 (+ 3784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:12:57.158746 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:12:57.158750 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:12:57.158786 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:12:57.161302 (+ 2516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:12:57.162056 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:12:57.162060 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:12:57.162063 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:12:57.162189 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:12:57.162673 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:12:57.162677 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:12:57.162679 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:12:57.162970 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:12:57.163122 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0811 09:12:57.163885 (+ 763us) server_negotiation.cc:299] Negotiation successful
0811 09:12:57.164129 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":235,"threads_started":1}
I20260811 09:12:57.165668 31051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:57.147028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:36913 (local address 127.28.154.129:49855)
0811 09:12:57.147875 (+ 847us) negotiation.cc:107] Waiting for socket to connect
0811 09:12:57.147925 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 09:12:57.149799 (+ 1874us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:12:57.150581 (+ 782us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:12:57.150590 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:12:57.151235 (+ 645us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:12:57.151917 (+ 682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:12:57.151939 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:12:57.153024 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:12:57.153028 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:12:57.154285 (+ 1257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:12:57.154299 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:12:57.155080 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:12:57.155088 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:12:57.155320 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:12:57.156013 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:12:57.156042 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:12:57.158498 (+ 2456us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:12:57.161456 (+ 2958us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:12:57.161467 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:12:57.161484 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:12:57.161932 (+ 448us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:12:57.162301 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:12:57.162305 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:12:57.162309 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:12:57.162556 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:12:57.163103 (+ 547us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:12:57.163120 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:12:57.163489 (+ 369us) client_negotiation.cc:769] Sending connection context
0811 09:12:57.163748 (+ 259us) client_negotiation.cc:240] Negotiation successful
0811 09:12:57.164315 (+ 567us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":517,"thread_start_us":313,"threads_started":1}
I20260811 09:12:57.167289 31044 heartbeater.cc:347] Connected to a master server at 127.28.154.190:36913
I20260811 09:12:57.167636 31044 heartbeater.cc:464] Registering TS with master...
I20260811 09:12:57.168509 31044 heartbeater.cc:511] Master 127.28.154.190:36913 requested a full tablet report, sending...
I20260811 09:12:57.170765 30687 ts_manager.cc:194] Registered new tserver with Master: 6b27f3f959b04fee804d7c32e03866d2 (127.28.154.129:46333)
I20260811 09:12:57.172188 30687 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.154.129@KRBTEST.COM'} at 127.28.154.129:49855
WARNING: no policy specified for kudu/127.28.154.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.154.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.154.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.130.keytab.
Entry for principal kudu/127.28.154.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.130.keytab.
I20260811 09:12:57.219727 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439250192320-29290-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439250192320-29290-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439250192320-29290-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439250192320-29290-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.154.130:0
--local_ip_for_outbound_sockets=127.28.154.130
--webserver_interface=127.28.154.130
--webserver_port=0
--tserver_master_addrs=127.28.154.190:36913
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.130.keytab
--principal=kudu/127.28.154.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:44533
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:12:57.557821 31055 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:12:57.558141 31055 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:12:57.558284 31055 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:12:57.565583 31055 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:12:57.565727 31055 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:12:57.565892 31055 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.154.130
I20260811 09:12:57.575807 31055 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:44533
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439250192320-29290-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439250192320-29290-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.130.keytab
--principal=kudu/127.28.154.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.154.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439250192320-29290-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.154.130
--webserver_port=0
--tserver_master_addrs=127.28.154.190:36913
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.154.130
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439250192320-29290-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:12:57.577179 31055 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:12:57.578459 31055 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:12:57.591105 31066 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:12:57.591069 31068 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:12:57.591126 31071 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:12:57.592525 31055 server_base.cc:1034] running on GCE node
I20260811 09:12:57.593154 31055 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:12:57.594117 31055 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:12:57.595335 31055 hybrid_clock.cc:648] HybridClock initialized: now 1786439577595288 us; error 42 us; skew 500 ppm
I20260811 09:12:57.595643 31055 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:12:57 dist-test-slave-xt4k krb5kdc[30588](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439577, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:12:57.598855 31055 init.cc:377] Logged in from keytab as kudu/127.28.154.130@KRBTEST.COM (short username kudu)
I20260811 09:12:57.600306 31055 webserver.cc:460] Webserver started at http://127.28.154.130:42325/ using document root <none> and password file <none>
I20260811 09:12:57.600992 31055 fs_manager.cc:359] Metadata directory not provided
I20260811 09:12:57.601094 31055 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:12:57.601370 31055 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:12:57.604182 31055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439250192320-29290-0/minicluster-data/ts-1/data/instance:
uuid: "72b308ae3c1d4b38b08d35f36de1887c"
format_stamp: "Formatted at 2026-08-11 09:12:57 on dist-test-slave-xt4k"
server_key: "5f5db2b9af212907fd8cead2738b0024"
server_key_iv: "fe62bfbb997f9e581ecdc2bc9323706f"
server_key_version: "encryptionkey@0"
I20260811 09:12:57.604938 31055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439250192320-29290-0/minicluster-data/ts-1/wal/instance:
uuid: "72b308ae3c1d4b38b08d35f36de1887c"
format_stamp: "Formatted at 2026-08-11 09:12:57 on dist-test-slave-xt4k"
server_key: "5f5db2b9af212907fd8cead2738b0024"
server_key_iv: "fe62bfbb997f9e581ecdc2bc9323706f"
server_key_version: "encryptionkey@0"
I20260811 09:12:57.610036 31055 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 09:12:57.613371 31084 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:12:57.614650 31055 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 09:12:57.614792 31055 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439250192320-29290-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439250192320-29290-0/minicluster-data/ts-1/wal
uuid: "72b308ae3c1d4b38b08d35f36de1887c"
format_stamp: "Formatted at 2026-08-11 09:12:57 on dist-test-slave-xt4k"
server_key: "5f5db2b9af212907fd8cead2738b0024"
server_key_iv: "fe62bfbb997f9e581ecdc2bc9323706f"
server_key_version: "encryptionkey@0"
I20260811 09:12:57.614976 31055 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439250192320-29290-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439250192320-29290-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439250192320-29290-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:12:57.633787 31055 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:12:57.638046 31055 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:12:57.638316 31055 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:12:57.639267 31055 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:12:57.640645 31055 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:12:57.640718 31055 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:12:57.640812 31055 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:12:57.640856 31055 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:12:57.669835 31055 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.130:43717
I20260811 09:12:57.669880 31294 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.130:43717 every 8 connection(s)
I20260811 09:12:57.671118 31055 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439250192320-29290-0/minicluster-data/ts-1/data/info.pb
I20260811 09:12:57.681106 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 31055
I20260811 09:12:57.681320 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439250192320-29290-0/minicluster-data/ts-1/wal/instance
I20260811 09:12:57.681886 29290 external_mini_cluster.cc:1428] Setting key 75779893850b032dd7a6c0f859a12a0e
Aug 11 09:12:57 dist-test-slave-xt4k krb5kdc[30588](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439577, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.130@KRBTEST.COM for kudu/127.28.154.190@KRBTEST.COM
I20260811 09:12:57.692919 31302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:57.673970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.130:51401 (local address 127.28.154.190:36913)
0811 09:12:57.674414 (+ 444us) server_negotiation.cc:206] Beginning negotiation
0811 09:12:57.674420 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:12:57.675435 (+ 1015us) server_negotiation.cc:407] Connection header received
0811 09:12:57.677374 (+ 1939us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:12:57.677379 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:12:57.677463 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:12:57.677562 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:12:57.679126 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:57.679837 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:57.681852 (+ 2015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:57.682338 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:57.682517 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:12:57.686229 (+ 3712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:12:57.686236 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:12:57.686240 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:12:57.686281 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:12:57.689377 (+ 3096us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:12:57.690284 (+ 907us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:12:57.690290 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:12:57.690294 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:12:57.690455 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:12:57.690966 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:12:57.690970 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:12:57.690973 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:12:57.691319 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:12:57.691465 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0811 09:12:57.692265 (+ 800us) server_negotiation.cc:299] Negotiation successful
0811 09:12:57.692535 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":331,"thread_start_us":218,"threads_started":1}
I20260811 09:12:57.693830 31303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:57.674498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:36913 (local address 127.28.154.130:51401)
0811 09:12:57.675253 (+ 755us) negotiation.cc:107] Waiting for socket to connect
0811 09:12:57.675293 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 09:12:57.677099 (+ 1806us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:12:57.677818 (+ 719us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:12:57.677826 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:12:57.678352 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:12:57.678941 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:12:57.678958 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:12:57.679991 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:12:57.679995 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:12:57.681670 (+ 1675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:12:57.681685 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:12:57.682634 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:12:57.682644 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:12:57.682875 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:12:57.683636 (+ 761us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:12:57.683674 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:12:57.685961 (+ 2287us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:12:57.689557 (+ 3596us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:12:57.689566 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:12:57.689582 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:12:57.690142 (+ 560us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:12:57.690604 (+ 462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:12:57.690608 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:12:57.690610 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:12:57.690839 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:12:57.691472 (+ 633us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:12:57.691480 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:12:57.691857 (+ 377us) client_negotiation.cc:769] Sending connection context
0811 09:12:57.692126 (+ 269us) client_negotiation.cc:240] Negotiation successful
0811 09:12:57.692608 (+ 482us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":290,"threads_started":1}
I20260811 09:12:57.695345 31295 heartbeater.cc:347] Connected to a master server at 127.28.154.190:36913
I20260811 09:12:57.695662 31295 heartbeater.cc:464] Registering TS with master...
I20260811 09:12:57.696435 31295 heartbeater.cc:511] Master 127.28.154.190:36913 requested a full tablet report, sending...
I20260811 09:12:57.697971 30687 ts_manager.cc:194] Registered new tserver with Master: 72b308ae3c1d4b38b08d35f36de1887c (127.28.154.130:43717)
I20260811 09:12:57.699004 30687 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.154.130@KRBTEST.COM'} at 127.28.154.130:51401
WARNING: no policy specified for kudu/127.28.154.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.154.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.154.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.131.keytab.
Entry for principal kudu/127.28.154.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.131.keytab.
I20260811 09:12:57.751955 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439250192320-29290-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439250192320-29290-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439250192320-29290-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439250192320-29290-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.154.131:0
--local_ip_for_outbound_sockets=127.28.154.131
--webserver_interface=127.28.154.131
--webserver_port=0
--tserver_master_addrs=127.28.154.190:36913
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.131.keytab
--principal=kudu/127.28.154.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:44533
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:12:58.099889 31307 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:12:58.100216 31307 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:12:58.100378 31307 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:12:58.107838 31307 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:12:58.107986 31307 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:12:58.108145 31307 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.154.131
I20260811 09:12:58.119014 31307 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:44533
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439250192320-29290-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439250192320-29290-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.131.keytab
--principal=kudu/127.28.154.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.154.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439250192320-29290-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.154.131
--webserver_port=0
--tserver_master_addrs=127.28.154.190:36913
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.154.131
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439250192320-29290-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:12:58.120463 31307 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:12:58.121819 31307 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:12:58.135293 31313 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:12:58.135342 31316 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:12:58.135352 31314 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:12:58.136304 31307 server_base.cc:1034] running on GCE node
I20260811 09:12:58.136965 31307 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:12:58.137939 31307 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:12:58.139163 31307 hybrid_clock.cc:648] HybridClock initialized: now 1786439578139107 us; error 48 us; skew 500 ppm
I20260811 09:12:58.139484 31307 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:12:58 dist-test-slave-xt4k krb5kdc[30588](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439578, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:12:58.142908 31307 init.cc:377] Logged in from keytab as kudu/127.28.154.131@KRBTEST.COM (short username kudu)
I20260811 09:12:58.144536 31307 webserver.cc:460] Webserver started at http://127.28.154.131:33015/ using document root <none> and password file <none>
I20260811 09:12:58.145248 31307 fs_manager.cc:359] Metadata directory not provided
I20260811 09:12:58.145370 31307 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:12:58.145682 31307 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:12:58.148640 31307 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439250192320-29290-0/minicluster-data/ts-2/data/instance:
uuid: "bf995efc786f49349ca491bb5418c2e5"
format_stamp: "Formatted at 2026-08-11 09:12:58 on dist-test-slave-xt4k"
server_key: "2b348ff40574effe54a048138a5c26fd"
server_key_iv: "ed798f8a4a2921ab234d668a2f28fb9b"
server_key_version: "encryptionkey@0"
I20260811 09:12:58.149405 31307 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439250192320-29290-0/minicluster-data/ts-2/wal/instance:
uuid: "bf995efc786f49349ca491bb5418c2e5"
format_stamp: "Formatted at 2026-08-11 09:12:58 on dist-test-slave-xt4k"
server_key: "2b348ff40574effe54a048138a5c26fd"
server_key_iv: "ed798f8a4a2921ab234d668a2f28fb9b"
server_key_version: "encryptionkey@0"
I20260811 09:12:58.154596 31307 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 09:12:58.158656 31331 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:12:58.160266 31307 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.002s
I20260811 09:12:58.160470 31307 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439250192320-29290-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439250192320-29290-0/minicluster-data/ts-2/wal
uuid: "bf995efc786f49349ca491bb5418c2e5"
format_stamp: "Formatted at 2026-08-11 09:12:58 on dist-test-slave-xt4k"
server_key: "2b348ff40574effe54a048138a5c26fd"
server_key_iv: "ed798f8a4a2921ab234d668a2f28fb9b"
server_key_version: "encryptionkey@0"
I20260811 09:12:58.160686 31307 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439250192320-29290-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439250192320-29290-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439250192320-29290-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:12:58.175707 31044 heartbeater.cc:503] Master 127.28.154.190:36913 was elected leader, sending a full tablet report...
I20260811 09:12:58.198110 31307 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:12:58.202731 31307 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:12:58.203013 31307 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:12:58.204015 31307 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:12:58.205497 31307 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:12:58.205590 31307 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:12:58.205722 31307 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:12:58.205776 31307 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:12:58.239073 31307 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.131:36201
I20260811 09:12:58.239131 31527 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.131:36201 every 8 connection(s)
I20260811 09:12:58.240463 31307 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439250192320-29290-0/minicluster-data/ts-2/data/info.pb
I20260811 09:12:58.244819 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 31307
I20260811 09:12:58.245086 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439250192320-29290-0/minicluster-data/ts-2/wal/instance
I20260811 09:12:58.245668 29290 external_mini_cluster.cc:1428] Setting key 011ea5de2f5ec5d47e8a6239a0760cd7
Aug 11 09:12:58 dist-test-slave-xt4k krb5kdc[30588](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439578, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.131@KRBTEST.COM for kudu/127.28.154.190@KRBTEST.COM
I20260811 09:12:58.262583 31532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:58.243275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.131:45463 (local address 127.28.154.190:36913)
0811 09:12:58.243821 (+ 546us) server_negotiation.cc:206] Beginning negotiation
0811 09:12:58.243826 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:12:58.245062 (+ 1236us) server_negotiation.cc:407] Connection header received
0811 09:12:58.247423 (+ 2361us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:12:58.247427 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:12:58.247517 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:12:58.247633 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:12:58.249547 (+ 1914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:58.250364 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:58.252138 (+ 1774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:58.252600 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:58.252707 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:12:58.256154 (+ 3447us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:12:58.256158 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:12:58.256162 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:12:58.256197 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:12:58.258998 (+ 2801us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:12:58.259790 (+ 792us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:12:58.259797 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:12:58.259800 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:12:58.259931 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:12:58.260457 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:12:58.260463 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:12:58.260466 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:12:58.260864 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:12:58.261022 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0811 09:12:58.261908 (+ 886us) server_negotiation.cc:299] Negotiation successful
0811 09:12:58.262152 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":433,"thread_start_us":296,"threads_started":1}
I20260811 09:12:58.263567 31533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:58.243912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:36913 (local address 127.28.154.131:45463)
0811 09:12:58.244871 (+ 959us) negotiation.cc:107] Waiting for socket to connect
0811 09:12:58.244936 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0811 09:12:58.247144 (+ 2208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:12:58.247956 (+ 812us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:12:58.247971 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:12:58.248608 (+ 637us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:12:58.249359 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:12:58.249384 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:12:58.250582 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:12:58.250587 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:12:58.251991 (+ 1404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:12:58.252011 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:12:58.252832 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:12:58.252838 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:12:58.253056 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:12:58.253701 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:12:58.253729 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:12:58.255929 (+ 2200us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:12:58.259165 (+ 3236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:12:58.259175 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:12:58.259193 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:12:58.259663 (+ 470us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:12:58.260053 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:12:58.260058 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:12:58.260061 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:12:58.260357 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:12:58.260999 (+ 642us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:12:58.261007 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:12:58.261446 (+ 439us) client_negotiation.cc:769] Sending connection context
0811 09:12:58.261778 (+ 332us) client_negotiation.cc:240] Negotiation successful
0811 09:12:58.262355 (+ 577us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":639,"thread_start_us":352,"threads_started":1}
I20260811 09:12:58.265158 31528 heartbeater.cc:347] Connected to a master server at 127.28.154.190:36913
I20260811 09:12:58.265511 31528 heartbeater.cc:464] Registering TS with master...
I20260811 09:12:58.266278 31528 heartbeater.cc:511] Master 127.28.154.190:36913 requested a full tablet report, sending...
I20260811 09:12:58.267740 30687 ts_manager.cc:194] Registered new tserver with Master: bf995efc786f49349ca491bb5418c2e5 (127.28.154.131:36201)
I20260811 09:12:58.268649 30687 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.154.131@KRBTEST.COM'} at 127.28.154.131:45463
I20260811 09:12:58.274468 29290 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 09:12:58.292538 31532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:58.278667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54484 (local address 127.28.154.190:36913)
0811 09:12:58.278852 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 09:12:58.278857 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:12:58.279301 (+ 444us) server_negotiation.cc:407] Connection header received
0811 09:12:58.279419 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:12:58.279423 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:12:58.279506 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:12:58.279596 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:12:58.281140 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:58.281904 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:58.283318 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:58.283740 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:58.283842 (+ 102us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:12:58.286623 (+ 2781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:12:58.286628 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:12:58.286631 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:12:58.286667 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:12:58.289470 (+ 2803us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:12:58.290419 (+ 949us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:12:58.290423 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:12:58.290426 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:12:58.290574 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:12:58.291048 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:12:58.291051 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:12:58.291054 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:12:58.291443 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:12:58.291652 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0811 09:12:58.291993 (+ 341us) server_negotiation.cc:299] Negotiation successful
0811 09:12:58.292144 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 09:12:58.296856 30687 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54484:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 09:12:58.303797 30687 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 09:12:58.327612 31550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:58.316856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47688 (local address 127.28.154.130:43717)
0811 09:12:58.317479 (+ 623us) server_negotiation.cc:206] Beginning negotiation
0811 09:12:58.317489 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 09:12:58.318208 (+ 719us) server_negotiation.cc:407] Connection header received
0811 09:12:58.318675 (+ 467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:12:58.318684 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:12:58.318958 (+ 274us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:12:58.319131 (+ 173us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:12:58.320470 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:58.322204 (+ 1734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:58.324969 (+ 2765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:58.326282 (+ 1313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:58.326437 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:12:58.326651 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0811 09:12:58.327084 (+ 433us) server_negotiation.cc:299] Negotiation successful
0811 09:12:58.327255 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":449,"thread_start_us":585,"threads_started":1}
I20260811 09:12:58.328024 31552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:58.316661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.130:43717 (local address 127.0.0.1:47688)
0811 09:12:58.318039 (+ 1378us) negotiation.cc:107] Waiting for socket to connect
0811 09:12:58.318083 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 09:12:58.318374 (+ 291us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:12:58.319471 (+ 1097us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:12:58.319480 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:12:58.319514 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:12:58.320246 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:12:58.320263 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:12:58.322401 (+ 2138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:12:58.322407 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:12:58.324788 (+ 2381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:12:58.324805 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:12:58.326517 (+ 1712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:12:58.326524 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:12:58.326840 (+ 316us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:12:58.326886 (+ 46us) client_negotiation.cc:769] Sending connection context
0811 09:12:58.327020 (+ 134us) client_negotiation.cc:240] Negotiation successful
0811 09:12:58.327233 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1105,"mutex_wait_us":103,"thread_start_us":3268,"threads_started":1}
I20260811 09:12:58.330607 31547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:58.317387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.131:36201 (local address 127.0.0.1:48700)
0811 09:12:58.320207 (+ 2820us) negotiation.cc:107] Waiting for socket to connect
0811 09:12:58.320224 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 09:12:58.320356 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:12:58.323067 (+ 2711us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:12:58.323071 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:12:58.323103 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:12:58.323655 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:12:58.323670 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:12:58.325470 (+ 1800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:12:58.325476 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:12:58.327745 (+ 2269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:12:58.327762 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:12:58.329779 (+ 2017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:12:58.329785 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:12:58.329961 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:12:58.329993 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 09:12:58.330081 (+ 88us) client_negotiation.cc:240] Negotiation successful
0811 09:12:58.330241 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2697,"thread_start_us":1891,"threads_started":1}
I20260811 09:12:58.330647 31557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:58.317113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.129:46333 (local address 127.0.0.1:53376)
0811 09:12:58.319550 (+ 2437us) negotiation.cc:107] Waiting for socket to connect
0811 09:12:58.319568 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 09:12:58.319697 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:12:58.323355 (+ 3658us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:12:58.323359 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:12:58.323387 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:12:58.323923 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:12:58.323936 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:12:58.325790 (+ 1854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:12:58.325798 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:12:58.327905 (+ 2107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:12:58.327917 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:12:58.329691 (+ 1774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:12:58.329698 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:12:58.329879 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:12:58.329913 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 09:12:58.329984 (+ 71us) client_negotiation.cc:240] Negotiation successful
0811 09:12:58.330119 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2304,"mutex_wait_us":2,"thread_start_us":1420,"threads_started":1}
I20260811 09:12:58.331650 31556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:58.317288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53376 (local address 127.28.154.129:46333)
0811 09:12:58.322769 (+ 5481us) server_negotiation.cc:206] Beginning negotiation
0811 09:12:58.322777 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:12:58.322802 (+ 25us) server_negotiation.cc:407] Connection header received
0811 09:12:58.322889 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:12:58.322896 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:12:58.323119 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:12:58.323355 (+ 236us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:12:58.324101 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:58.325624 (+ 1523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:58.328070 (+ 2446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:58.329428 (+ 1358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:58.330918 (+ 1490us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:12:58.331093 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0811 09:12:58.331170 (+ 77us) server_negotiation.cc:299] Negotiation successful
0811 09:12:58.331337 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5320,"thread_start_us":5169,"threads_started":1}
I20260811 09:12:58.331705 31558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:58.317999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48700 (local address 127.28.154.131:36201)
0811 09:12:58.322412 (+ 4413us) server_negotiation.cc:206] Beginning negotiation
0811 09:12:58.322421 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 09:12:58.322456 (+ 35us) server_negotiation.cc:407] Connection header received
0811 09:12:58.322564 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:12:58.322572 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:12:58.322855 (+ 283us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:12:58.323052 (+ 197us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:12:58.323908 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:58.325324 (+ 1416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:58.327954 (+ 2630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:58.329448 (+ 1494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:58.331012 (+ 1564us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:12:58.331193 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0811 09:12:58.331263 (+ 70us) server_negotiation.cc:299] Negotiation successful
0811 09:12:58.331422 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4225,"thread_start_us":4066,"threads_started":1}
I20260811 09:12:58.332492 31178 tablet_service.cc:1467] Processing CreateTablet for tablet b68719ba18b34364be89b3bd012c1d3b (DEFAULT_TABLE table=test-table [id=bf80fe301c0749e3a050cbc79ee6ab03]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:12:58.334112 31178 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b68719ba18b34364be89b3bd012c1d3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:12:58.335345 30919 tablet_service.cc:1467] Processing CreateTablet for tablet b68719ba18b34364be89b3bd012c1d3b (DEFAULT_TABLE table=test-table [id=bf80fe301c0749e3a050cbc79ee6ab03]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:12:58.335381 31422 tablet_service.cc:1467] Processing CreateTablet for tablet b68719ba18b34364be89b3bd012c1d3b (DEFAULT_TABLE table=test-table [id=bf80fe301c0749e3a050cbc79ee6ab03]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:12:58.336854 30919 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b68719ba18b34364be89b3bd012c1d3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:12:58.336863 31422 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b68719ba18b34364be89b3bd012c1d3b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:12:58.347797 31559 tablet_bootstrap.cc:492] T b68719ba18b34364be89b3bd012c1d3b P 6b27f3f959b04fee804d7c32e03866d2: Bootstrap starting.
I20260811 09:12:58.351531 31559 tablet_bootstrap.cc:654] T b68719ba18b34364be89b3bd012c1d3b P 6b27f3f959b04fee804d7c32e03866d2: Neither blocks nor log segments found. Creating new log.
I20260811 09:12:58.352641 31559 log.cc:824] T b68719ba18b34364be89b3bd012c1d3b P 6b27f3f959b04fee804d7c32e03866d2: Log is configured to *not* fsync() on all Append() calls
I20260811 09:12:58.355532 31559 tablet_bootstrap.cc:492] T b68719ba18b34364be89b3bd012c1d3b P 6b27f3f959b04fee804d7c32e03866d2: No bootstrap required, opened a new log
I20260811 09:12:58.355789 31559 ts_tablet_manager.cc:1397] T b68719ba18b34364be89b3bd012c1d3b P 6b27f3f959b04fee804d7c32e03866d2: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260811 09:12:58.356581 31561 tablet_bootstrap.cc:492] T b68719ba18b34364be89b3bd012c1d3b P 72b308ae3c1d4b38b08d35f36de1887c: Bootstrap starting.
I20260811 09:12:58.360706 31561 tablet_bootstrap.cc:654] T b68719ba18b34364be89b3bd012c1d3b P 72b308ae3c1d4b38b08d35f36de1887c: Neither blocks nor log segments found. Creating new log.
I20260811 09:12:58.362208 31561 log.cc:824] T b68719ba18b34364be89b3bd012c1d3b P 72b308ae3c1d4b38b08d35f36de1887c: Log is configured to *not* fsync() on all Append() calls
I20260811 09:12:58.363229 31562 tablet_bootstrap.cc:492] T b68719ba18b34364be89b3bd012c1d3b P bf995efc786f49349ca491bb5418c2e5: Bootstrap starting.
I20260811 09:12:58.364449 31559 raft_consensus.cc:348] T b68719ba18b34364be89b3bd012c1d3b P 6b27f3f959b04fee804d7c32e03866d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72b308ae3c1d4b38b08d35f36de1887c" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 43717 } } peers { permanent_uuid: "6b27f3f959b04fee804d7c32e03866d2" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 46333 } } peers { permanent_uuid: "bf995efc786f49349ca491bb5418c2e5" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 36201 } }
I20260811 09:12:58.364959 31559 raft_consensus.cc:374] T b68719ba18b34364be89b3bd012c1d3b P 6b27f3f959b04fee804d7c32e03866d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:12:58.365091 31559 raft_consensus.cc:729] T b68719ba18b34364be89b3bd012c1d3b P 6b27f3f959b04fee804d7c32e03866d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b27f3f959b04fee804d7c32e03866d2, State: Initialized, Role: FOLLOWER
I20260811 09:12:58.365748 31559 consensus_queue.cc:260] T b68719ba18b34364be89b3bd012c1d3b P 6b27f3f959b04fee804d7c32e03866d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72b308ae3c1d4b38b08d35f36de1887c" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 43717 } } peers { permanent_uuid: "6b27f3f959b04fee804d7c32e03866d2" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 46333 } } peers { permanent_uuid: "bf995efc786f49349ca491bb5418c2e5" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 36201 } }
I20260811 09:12:58.367033 31562 tablet_bootstrap.cc:654] T b68719ba18b34364be89b3bd012c1d3b P bf995efc786f49349ca491bb5418c2e5: Neither blocks nor log segments found. Creating new log.
I20260811 09:12:58.368315 31562 log.cc:824] T b68719ba18b34364be89b3bd012c1d3b P bf995efc786f49349ca491bb5418c2e5: Log is configured to *not* fsync() on all Append() calls
I20260811 09:12:58.368984 31559 ts_tablet_manager.cc:1428] T b68719ba18b34364be89b3bd012c1d3b P 6b27f3f959b04fee804d7c32e03866d2: Time spent starting tablet: real 0.013s user 0.012s sys 0.001s
I20260811 09:12:58.372102 31561 tablet_bootstrap.cc:492] T b68719ba18b34364be89b3bd012c1d3b P 72b308ae3c1d4b38b08d35f36de1887c: No bootstrap required, opened a new log
I20260811 09:12:58.372447 31561 ts_tablet_manager.cc:1397] T b68719ba18b34364be89b3bd012c1d3b P 72b308ae3c1d4b38b08d35f36de1887c: Time spent bootstrapping tablet: real 0.019s user 0.008s sys 0.008s
I20260811 09:12:58.374009 31562 tablet_bootstrap.cc:492] T b68719ba18b34364be89b3bd012c1d3b P bf995efc786f49349ca491bb5418c2e5: No bootstrap required, opened a new log
I20260811 09:12:58.374293 31562 ts_tablet_manager.cc:1397] T b68719ba18b34364be89b3bd012c1d3b P bf995efc786f49349ca491bb5418c2e5: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.004s
I20260811 09:12:58.383200 31561 raft_consensus.cc:348] T b68719ba18b34364be89b3bd012c1d3b P 72b308ae3c1d4b38b08d35f36de1887c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72b308ae3c1d4b38b08d35f36de1887c" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 43717 } } peers { permanent_uuid: "6b27f3f959b04fee804d7c32e03866d2" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 46333 } } peers { permanent_uuid: "bf995efc786f49349ca491bb5418c2e5" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 36201 } }
I20260811 09:12:58.383730 31561 raft_consensus.cc:374] T b68719ba18b34364be89b3bd012c1d3b P 72b308ae3c1d4b38b08d35f36de1887c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:12:58.383837 31561 raft_consensus.cc:729] T b68719ba18b34364be89b3bd012c1d3b P 72b308ae3c1d4b38b08d35f36de1887c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72b308ae3c1d4b38b08d35f36de1887c, State: Initialized, Role: FOLLOWER
I20260811 09:12:58.384539 31561 consensus_queue.cc:260] T b68719ba18b34364be89b3bd012c1d3b P 72b308ae3c1d4b38b08d35f36de1887c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72b308ae3c1d4b38b08d35f36de1887c" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 43717 } } peers { permanent_uuid: "6b27f3f959b04fee804d7c32e03866d2" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 46333 } } peers { permanent_uuid: "bf995efc786f49349ca491bb5418c2e5" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 36201 } }
I20260811 09:12:58.384608 31562 raft_consensus.cc:348] T b68719ba18b34364be89b3bd012c1d3b P bf995efc786f49349ca491bb5418c2e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72b308ae3c1d4b38b08d35f36de1887c" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 43717 } } peers { permanent_uuid: "6b27f3f959b04fee804d7c32e03866d2" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 46333 } } peers { permanent_uuid: "bf995efc786f49349ca491bb5418c2e5" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 36201 } }
I20260811 09:12:58.385031 31562 raft_consensus.cc:374] T b68719ba18b34364be89b3bd012c1d3b P bf995efc786f49349ca491bb5418c2e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:12:58.385142 31562 raft_consensus.cc:729] T b68719ba18b34364be89b3bd012c1d3b P bf995efc786f49349ca491bb5418c2e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf995efc786f49349ca491bb5418c2e5, State: Initialized, Role: FOLLOWER
I20260811 09:12:58.385702 31295 heartbeater.cc:503] Master 127.28.154.190:36913 was elected leader, sending a full tablet report...
I20260811 09:12:58.385735 31562 consensus_queue.cc:260] T b68719ba18b34364be89b3bd012c1d3b P bf995efc786f49349ca491bb5418c2e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72b308ae3c1d4b38b08d35f36de1887c" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 43717 } } peers { permanent_uuid: "6b27f3f959b04fee804d7c32e03866d2" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 46333 } } peers { permanent_uuid: "bf995efc786f49349ca491bb5418c2e5" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 36201 } }
I20260811 09:12:58.386524 31561 ts_tablet_manager.cc:1428] T b68719ba18b34364be89b3bd012c1d3b P 72b308ae3c1d4b38b08d35f36de1887c: Time spent starting tablet: real 0.014s user 0.008s sys 0.004s
I20260811 09:12:58.387172 31528 heartbeater.cc:503] Master 127.28.154.190:36913 was elected leader, sending a full tablet report...
I20260811 09:12:58.387322 31562 ts_tablet_manager.cc:1428] T b68719ba18b34364be89b3bd012c1d3b P bf995efc786f49349ca491bb5418c2e5: Time spent starting tablet: real 0.013s user 0.010s sys 0.004s
W20260811 09:12:58.396064 31046 tablet.cc:2369] T b68719ba18b34364be89b3bd012c1d3b P 6b27f3f959b04fee804d7c32e03866d2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 09:12:58.423271 31298 tablet.cc:2369] T b68719ba18b34364be89b3bd012c1d3b P 72b308ae3c1d4b38b08d35f36de1887c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 09:12:58.474500 31570 raft_consensus.cc:482] T b68719ba18b34364be89b3bd012c1d3b P 6b27f3f959b04fee804d7c32e03866d2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:12:58.474831 31570 raft_consensus.cc:504] T b68719ba18b34364be89b3bd012c1d3b P 6b27f3f959b04fee804d7c32e03866d2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72b308ae3c1d4b38b08d35f36de1887c" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 43717 } } peers { permanent_uuid: "6b27f3f959b04fee804d7c32e03866d2" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 46333 } } peers { permanent_uuid: "bf995efc786f49349ca491bb5418c2e5" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 36201 } }
I20260811 09:12:58.476934 31570 leader_election.cc:288] T b68719ba18b34364be89b3bd012c1d3b P 6b27f3f959b04fee804d7c32e03866d2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 72b308ae3c1d4b38b08d35f36de1887c (127.28.154.130:43717), bf995efc786f49349ca491bb5418c2e5 (127.28.154.131:36201)
I20260811 09:12:58.485265 31550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:58.477450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.129:57609 (local address 127.28.154.130:43717)
0811 09:12:58.477622 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 09:12:58.477626 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:12:58.479230 (+ 1604us) server_negotiation.cc:407] Connection header received
0811 09:12:58.479437 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:12:58.479443 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:12:58.479513 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:12:58.479562 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:12:58.480265 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:58.481345 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:58.483528 (+ 2183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:58.484296 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:58.484505 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:12:58.484584 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 09:12:58.484868 (+ 284us) server_negotiation.cc:299] Negotiation successful
0811 09:12:58.485007 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 09:12:58.485337 31601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:58.477942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.130:43717 (local address 127.28.154.129:57609)
0811 09:12:58.479113 (+ 1171us) negotiation.cc:107] Waiting for socket to connect
0811 09:12:58.479130 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 09:12:58.479253 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:12:58.479713 (+ 460us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:12:58.479716 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:12:58.479734 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:12:58.480096 (+ 362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:12:58.480108 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:12:58.481493 (+ 1385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:12:58.481499 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:12:58.483377 (+ 1878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:12:58.483389 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:12:58.484549 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:12:58.484556 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:12:58.484709 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:12:58.484737 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 09:12:58.484823 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 09:12:58.484965 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1060,"mutex_wait_us":3,"thread_start_us":807,"threads_started":1}
I20260811 09:12:58.485838 31558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:58.477354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.129:34325 (local address 127.28.154.131:36201)
0811 09:12:58.477632 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0811 09:12:58.477638 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:12:58.478833 (+ 1195us) server_negotiation.cc:407] Connection header received
0811 09:12:58.479133 (+ 300us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:12:58.479139 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:12:58.479226 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:12:58.479325 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:12:58.479883 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:58.480787 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:58.483139 (+ 2352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:58.484364 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:58.484526 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:12:58.484611 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 09:12:58.485126 (+ 515us) server_negotiation.cc:299] Negotiation successful
0811 09:12:58.485264 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 09:12:58.486467 31600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:58.477925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.131:36201 (local address 127.28.154.129:34325)
0811 09:12:58.478719 (+ 794us) negotiation.cc:107] Waiting for socket to connect
0811 09:12:58.478743 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 09:12:58.478968 (+ 225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:12:58.479342 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:12:58.479347 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:12:58.479373 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:12:58.479716 (+ 343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:12:58.479727 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:12:58.480928 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:12:58.480932 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:12:58.482972 (+ 2040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:12:58.482988 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:12:58.484761 (+ 1773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:12:58.484767 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:12:58.484946 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:12:58.484979 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 09:12:58.486011 (+ 1032us) client_negotiation.cc:240] Negotiation successful
0811 09:12:58.486153 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":652,"thread_start_us":436,"threads_started":1}
I20260811 09:12:58.486362 31214 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b68719ba18b34364be89b3bd012c1d3b" candidate_uuid: "6b27f3f959b04fee804d7c32e03866d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72b308ae3c1d4b38b08d35f36de1887c" is_pre_election: true
I20260811 09:12:58.486943 31214 raft_consensus.cc:2454] T b68719ba18b34364be89b3bd012c1d3b P 72b308ae3c1d4b38b08d35f36de1887c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b27f3f959b04fee804d7c32e03866d2 in term 0.
I20260811 09:12:58.487568 31455 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b68719ba18b34364be89b3bd012c1d3b" candidate_uuid: "6b27f3f959b04fee804d7c32e03866d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf995efc786f49349ca491bb5418c2e5" is_pre_election: true
I20260811 09:12:58.487713 30836 leader_election.cc:302] T b68719ba18b34364be89b3bd012c1d3b P 6b27f3f959b04fee804d7c32e03866d2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b27f3f959b04fee804d7c32e03866d2, 72b308ae3c1d4b38b08d35f36de1887c; no voters:
I20260811 09:12:58.488153 31570 raft_consensus.cc:2781] T b68719ba18b34364be89b3bd012c1d3b P 6b27f3f959b04fee804d7c32e03866d2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 09:12:58.488134 31455 raft_consensus.cc:2454] T b68719ba18b34364be89b3bd012c1d3b P bf995efc786f49349ca491bb5418c2e5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b27f3f959b04fee804d7c32e03866d2 in term 0.
I20260811 09:12:58.488313 31570 raft_consensus.cc:482] T b68719ba18b34364be89b3bd012c1d3b P 6b27f3f959b04fee804d7c32e03866d2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 09:12:58.488430 31570 raft_consensus.cc:3037] T b68719ba18b34364be89b3bd012c1d3b P 6b27f3f959b04fee804d7c32e03866d2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:12:58.490890 31570 raft_consensus.cc:504] T b68719ba18b34364be89b3bd012c1d3b P 6b27f3f959b04fee804d7c32e03866d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72b308ae3c1d4b38b08d35f36de1887c" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 43717 } } peers { permanent_uuid: "6b27f3f959b04fee804d7c32e03866d2" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 46333 } } peers { permanent_uuid: "bf995efc786f49349ca491bb5418c2e5" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 36201 } }
I20260811 09:12:58.491474 31570 leader_election.cc:288] T b68719ba18b34364be89b3bd012c1d3b P 6b27f3f959b04fee804d7c32e03866d2 [CANDIDATE]: Term 1 election: Requested vote from peers 72b308ae3c1d4b38b08d35f36de1887c (127.28.154.130:43717), bf995efc786f49349ca491bb5418c2e5 (127.28.154.131:36201)
I20260811 09:12:58.492026 31214 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b68719ba18b34364be89b3bd012c1d3b" candidate_uuid: "6b27f3f959b04fee804d7c32e03866d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72b308ae3c1d4b38b08d35f36de1887c"
I20260811 09:12:58.492030 31455 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b68719ba18b34364be89b3bd012c1d3b" candidate_uuid: "6b27f3f959b04fee804d7c32e03866d2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bf995efc786f49349ca491bb5418c2e5"
W20260811 09:12:58.492113 31530 tablet.cc:2369] T b68719ba18b34364be89b3bd012c1d3b P bf995efc786f49349ca491bb5418c2e5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 09:12:58.492259 31455 raft_consensus.cc:3037] T b68719ba18b34364be89b3bd012c1d3b P bf995efc786f49349ca491bb5418c2e5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:12:58.492252 31214 raft_consensus.cc:3037] T b68719ba18b34364be89b3bd012c1d3b P 72b308ae3c1d4b38b08d35f36de1887c [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:12:58.495458 31214 raft_consensus.cc:2454] T b68719ba18b34364be89b3bd012c1d3b P 72b308ae3c1d4b38b08d35f36de1887c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b27f3f959b04fee804d7c32e03866d2 in term 1.
I20260811 09:12:58.495461 31455 raft_consensus.cc:2454] T b68719ba18b34364be89b3bd012c1d3b P bf995efc786f49349ca491bb5418c2e5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b27f3f959b04fee804d7c32e03866d2 in term 1.
I20260811 09:12:58.495936 30834 leader_election.cc:302] T b68719ba18b34364be89b3bd012c1d3b P 6b27f3f959b04fee804d7c32e03866d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6b27f3f959b04fee804d7c32e03866d2, bf995efc786f49349ca491bb5418c2e5; no voters:
I20260811 09:12:58.496238 31570 raft_consensus.cc:2781] T b68719ba18b34364be89b3bd012c1d3b P 6b27f3f959b04fee804d7c32e03866d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:12:58.496683 31570 raft_consensus.cc:686] T b68719ba18b34364be89b3bd012c1d3b P 6b27f3f959b04fee804d7c32e03866d2 [term 1 LEADER]: Becoming Leader. State: Replica: 6b27f3f959b04fee804d7c32e03866d2, State: Running, Role: LEADER
I20260811 09:12:58.497164 31570 consensus_queue.cc:237] T b68719ba18b34364be89b3bd012c1d3b P 6b27f3f959b04fee804d7c32e03866d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72b308ae3c1d4b38b08d35f36de1887c" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 43717 } } peers { permanent_uuid: "6b27f3f959b04fee804d7c32e03866d2" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 46333 } } peers { permanent_uuid: "bf995efc786f49349ca491bb5418c2e5" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 36201 } }
I20260811 09:12:58.502058 30687 catalog_manager.cc:5495] T b68719ba18b34364be89b3bd012c1d3b P 6b27f3f959b04fee804d7c32e03866d2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6b27f3f959b04fee804d7c32e03866d2 (127.28.154.129). New cstate: current_term: 1 leader_uuid: "6b27f3f959b04fee804d7c32e03866d2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72b308ae3c1d4b38b08d35f36de1887c" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 43717 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b27f3f959b04fee804d7c32e03866d2" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 46333 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bf995efc786f49349ca491bb5418c2e5" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 36201 } health_report { overall_health: UNKNOWN } } }
I20260811 09:12:58.524597 31556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:58.518682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53378 (local address 127.28.154.129:46333)
0811 09:12:58.518932 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0811 09:12:58.518938 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:12:58.519076 (+ 138us) server_negotiation.cc:407] Connection header received
0811 09:12:58.519301 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:12:58.519305 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:12:58.519405 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:12:58.519535 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:12:58.520021 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:58.520797 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:58.522287 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:58.522716 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:58.522829 (+ 113us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:12:58.523477 (+ 648us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:12:58.523989 (+ 512us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:12:58.524149 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0811 09:12:58.524308 (+ 159us) server_negotiation.cc:299] Negotiation successful
0811 09:12:58.524378 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 09:12:58.532830 31455 raft_consensus.cc:1261] T b68719ba18b34364be89b3bd012c1d3b P bf995efc786f49349ca491bb5418c2e5 [term 1 FOLLOWER]: Refusing update from remote peer 6b27f3f959b04fee804d7c32e03866d2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:12:58.532835 31214 raft_consensus.cc:1261] T b68719ba18b34364be89b3bd012c1d3b P 72b308ae3c1d4b38b08d35f36de1887c [term 1 FOLLOWER]: Refusing update from remote peer 6b27f3f959b04fee804d7c32e03866d2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:12:58.533864 31603 consensus_queue.cc:1035] T b68719ba18b34364be89b3bd012c1d3b P 6b27f3f959b04fee804d7c32e03866d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "72b308ae3c1d4b38b08d35f36de1887c" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 43717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:12:58.534209 31570 consensus_queue.cc:1035] T b68719ba18b34364be89b3bd012c1d3b P 6b27f3f959b04fee804d7c32e03866d2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bf995efc786f49349ca491bb5418c2e5" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 36201 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:12:58.545274 31606 mvcc.cc:205] Tried to move back new op lower bound from 7317256513660428288 to 7317256513528102912. Current Snapshot: MvccSnapshot[applied={T|T < 7317256513660428288}]
I20260811 09:12:58.558262 31607 mvcc.cc:205] Tried to move back new op lower bound from 7317256513660428288 to 7317256513528102912. Current Snapshot: MvccSnapshot[applied={T|T < 7317256513660428288 or (T in {7317256513660428288})}]
I20260811 09:12:58.561013 30685 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54484:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:12:58.561308 30685 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54484:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:12:58.565182 30685 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ddd5f0813ca54f1388e68eed82828e48: Sending DeleteTablet for 3 replicas of tablet b68719ba18b34364be89b3bd012c1d3b
I20260811 09:12:58.566489 31178 tablet_service.cc:1514] Processing DeleteTablet for tablet b68719ba18b34364be89b3bd012c1d3b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:12:58 UTC) from {username='kudu', principal='kudu/127.28.154.190@KRBTEST.COM'} at 127.0.0.1:47688
I20260811 09:12:58.566489 31422 tablet_service.cc:1514] Processing DeleteTablet for tablet b68719ba18b34364be89b3bd012c1d3b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:12:58 UTC) from {username='kudu', principal='kudu/127.28.154.190@KRBTEST.COM'} at 127.0.0.1:48700
I20260811 09:12:58.566531 30919 tablet_service.cc:1514] Processing DeleteTablet for tablet b68719ba18b34364be89b3bd012c1d3b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:12:58 UTC) from {username='kudu', principal='kudu/127.28.154.190@KRBTEST.COM'} at 127.0.0.1:53376
I20260811 09:12:58.569094 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 30799
I20260811 09:12:58.601671 31634 tablet_replica.cc:330] T b68719ba18b34364be89b3bd012c1d3b P bf995efc786f49349ca491bb5418c2e5: stopping tablet replica
I20260811 09:12:58.602265 31634 raft_consensus.cc:2229] T b68719ba18b34364be89b3bd012c1d3b P bf995efc786f49349ca491bb5418c2e5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:12:58.602913 31634 raft_consensus.cc:2258] T b68719ba18b34364be89b3bd012c1d3b P bf995efc786f49349ca491bb5418c2e5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:12:58.603833 31633 tablet_replica.cc:330] T b68719ba18b34364be89b3bd012c1d3b P 6b27f3f959b04fee804d7c32e03866d2: stopping tablet replica
I20260811 09:12:58.604316 31633 raft_consensus.cc:2229] T b68719ba18b34364be89b3bd012c1d3b P 6b27f3f959b04fee804d7c32e03866d2 [term 1 LEADER]: Raft consensus shutting down.
I20260811 09:12:58.605113 31633 raft_consensus.cc:2258] T b68719ba18b34364be89b3bd012c1d3b P 6b27f3f959b04fee804d7c32e03866d2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:12:58.607719 31633 ts_tablet_manager.cc:1905] T b68719ba18b34364be89b3bd012c1d3b P 6b27f3f959b04fee804d7c32e03866d2: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 11 09:12:58 dist-test-slave-xt4k krb5kdc[30588](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439575, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.154.129@KRBTEST.COM
I20260811 09:12:58.612426 31630 tablet_replica.cc:330] T b68719ba18b34364be89b3bd012c1d3b P 72b308ae3c1d4b38b08d35f36de1887c: stopping tablet replica
I20260811 09:12:58.612830 31630 raft_consensus.cc:2229] T b68719ba18b34364be89b3bd012c1d3b P 72b308ae3c1d4b38b08d35f36de1887c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:12:58.613320 31630 raft_consensus.cc:2258] T b68719ba18b34364be89b3bd012c1d3b P 72b308ae3c1d4b38b08d35f36de1887c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:12:58.615170 31634 ts_tablet_manager.cc:1905] T b68719ba18b34364be89b3bd012c1d3b P bf995efc786f49349ca491bb5418c2e5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:12:58.615477 31630 ts_tablet_manager.cc:1905] T b68719ba18b34364be89b3bd012c1d3b P 72b308ae3c1d4b38b08d35f36de1887c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:12:58.615893 31633 ts_tablet_manager.cc:1918] T b68719ba18b34364be89b3bd012c1d3b P 6b27f3f959b04fee804d7c32e03866d2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:12:58.616108 31633 log.cc:1196] T b68719ba18b34364be89b3bd012c1d3b P 6b27f3f959b04fee804d7c32e03866d2: Deleting WAL directory at /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439250192320-29290-0/minicluster-data/ts-0/wal/wals/b68719ba18b34364be89b3bd012c1d3b
I20260811 09:12:58.616717 31633 ts_tablet_manager.cc:1939] T b68719ba18b34364be89b3bd012c1d3b P 6b27f3f959b04fee804d7c32e03866d2: Deleting consensus metadata
I20260811 09:12:58.618721 30659 catalog_manager.cc:4841] TS 6b27f3f959b04fee804d7c32e03866d2 (127.28.154.129:46333): tablet b68719ba18b34364be89b3bd012c1d3b (table test-table [id=bf80fe301c0749e3a050cbc79ee6ab03]) successfully deleted
I20260811 09:12:58.622771 31630 ts_tablet_manager.cc:1918] T b68719ba18b34364be89b3bd012c1d3b P 72b308ae3c1d4b38b08d35f36de1887c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:12:58.623021 31630 log.cc:1196] T b68719ba18b34364be89b3bd012c1d3b P 72b308ae3c1d4b38b08d35f36de1887c: Deleting WAL directory at /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439250192320-29290-0/minicluster-data/ts-1/wal/wals/b68719ba18b34364be89b3bd012c1d3b
I20260811 09:12:58.623643 31630 ts_tablet_manager.cc:1939] T b68719ba18b34364be89b3bd012c1d3b P 72b308ae3c1d4b38b08d35f36de1887c: Deleting consensus metadata
I20260811 09:12:58.625677 31634 ts_tablet_manager.cc:1918] T b68719ba18b34364be89b3bd012c1d3b P bf995efc786f49349ca491bb5418c2e5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:12:58.625735 30660 catalog_manager.cc:4841] TS 72b308ae3c1d4b38b08d35f36de1887c (127.28.154.130:43717): tablet b68719ba18b34364be89b3bd012c1d3b (table test-table [id=bf80fe301c0749e3a050cbc79ee6ab03]) successfully deleted
I20260811 09:12:58.625849 31634 log.cc:1196] T b68719ba18b34364be89b3bd012c1d3b P bf995efc786f49349ca491bb5418c2e5: Deleting WAL directory at /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786439250192320-29290-0/minicluster-data/ts-2/wal/wals/b68719ba18b34364be89b3bd012c1d3b
I20260811 09:12:58.626372 31634 ts_tablet_manager.cc:1939] T b68719ba18b34364be89b3bd012c1d3b P bf995efc786f49349ca491bb5418c2e5: Deleting consensus metadata
I20260811 09:12:58.626699 31556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:58.570782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53390 (local address 127.28.154.129:46333)
0811 09:12:58.571022 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0811 09:12:58.571029 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:12:58.593842 (+ 22813us) server_negotiation.cc:407] Connection header received
0811 09:12:58.594505 (+ 663us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:12:58.594513 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:12:58.594620 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:12:58.594709 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:12:58.596950 (+ 2241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:58.598300 (+ 1350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:58.605652 (+ 7352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:58.606436 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:58.606553 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:12:58.618791 (+ 12238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:12:58.618806 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:12:58.618825 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:12:58.618867 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:12:58.622918 (+ 4051us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:12:58.623974 (+ 1056us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:12:58.623984 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:12:58.623988 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:12:58.624200 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:12:58.624754 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:12:58.624759 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:12:58.624762 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:12:58.625276 (+ 514us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:12:58.625565 (+ 289us) server_negotiation.cc:1035] Waiting for connection context
0811 09:12:58.626200 (+ 635us) server_negotiation.cc:299] Negotiation successful
0811 09:12:58.626403 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260811 09:12:58.628086 31032 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:12:58.628142 30658 catalog_manager.cc:4841] TS bf995efc786f49349ca491bb5418c2e5 (127.28.154.131:36201): tablet b68719ba18b34364be89b3bd012c1d3b (table test-table [id=bf80fe301c0749e3a050cbc79ee6ab03]) successfully deleted
I20260811 09:12:58.729246 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 30799
I20260811 09:12:58.753211 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 31055
Aug 11 09:12:58 dist-test-slave-xt4k krb5kdc[30588](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439575, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.154.130@KRBTEST.COM
I20260811 09:12:58.768668 31550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:58.754055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47698 (local address 127.28.154.130:43717)
0811 09:12:58.754259 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 09:12:58.754265 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:12:58.754284 (+ 19us) server_negotiation.cc:407] Connection header received
0811 09:12:58.754346 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:12:58.754351 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:12:58.754413 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:12:58.754508 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:12:58.755815 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:58.756652 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:58.757928 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:58.758317 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:58.758428 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:12:58.762708 (+ 4280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:12:58.762724 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:12:58.762743 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:12:58.762779 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:12:58.765831 (+ 3052us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:12:58.766534 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:12:58.766541 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:12:58.766544 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:12:58.766680 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:12:58.767039 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:12:58.767043 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:12:58.767046 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:12:58.767447 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:12:58.767752 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0811 09:12:58.768050 (+ 298us) server_negotiation.cc:299] Negotiation successful
0811 09:12:58.768275 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 09:12:58.769408 31286 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:12:58.860750 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 31055
I20260811 09:12:58.884106 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 31307
Aug 11 09:12:58 dist-test-slave-xt4k krb5kdc[30588](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439575, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.154.131@KRBTEST.COM
I20260811 09:12:58.899370 31558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:58.884943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48710 (local address 127.28.154.131:36201)
0811 09:12:58.885147 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 09:12:58.885152 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:12:58.885174 (+ 22us) server_negotiation.cc:407] Connection header received
0811 09:12:58.885292 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:12:58.885296 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:12:58.885363 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:12:58.885450 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:12:58.886858 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:58.887627 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:58.888918 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:58.889317 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:58.889443 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:12:58.893784 (+ 4341us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:12:58.893801 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:12:58.893820 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:12:58.893859 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:12:58.896568 (+ 2709us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:12:58.897243 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:12:58.897251 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:12:58.897253 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:12:58.897408 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:12:58.897830 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:12:58.897833 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:12:58.897836 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:12:58.898227 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:12:58.898518 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0811 09:12:58.898816 (+ 298us) server_negotiation.cc:299] Negotiation successful
0811 09:12:58.899035 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 09:12:58.900072 31517 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:12:58.991214 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 31307
I20260811 09:12:59.015071 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 30624
I20260811 09:12:59.016196 30739 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:12:59.133525 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 30624
2026-08-11T09:12:59Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (4736 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-11T09:12:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:12:59Z Disabled control of system clock
I20260811 09:12:59.228165 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439250192320-29290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439250192320-29290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439250192320-29290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.154.190:37829
--webserver_interface=127.28.154.190
--webserver_port=0
--builtin_ntp_servers=127.28.154.148:34493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.154.190:37829
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 09:12:59.571446 31650 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:12:59.571736 31650 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:12:59.571848 31650 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:12:59.579187 31650 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:12:59.579301 31650 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:12:59.579344 31650 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:12:59.579427 31650 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:12:59.579492 31650 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:12:59.588862 31650 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:34493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439250192320-29290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439250192320-29290-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.154.190:37829
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.154.190:37829
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.154.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439250192320-29290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:12:59.590163 31650 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:12:59.591964 31650 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:12:59.603235 31658 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:12:59.603153 31659 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:12:59.603183 31667 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:12:59.604130 31650 server_base.cc:1034] running on GCE node
I20260811 09:12:59.604934 31650 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:12:59.606473 31650 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:12:59.607690 31650 hybrid_clock.cc:648] HybridClock initialized: now 1786439579607623 us; error 63 us; skew 500 ppm
I20260811 09:12:59.608060 31650 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 09:12:59.610338 31650 webserver.cc:460] Webserver started at http://127.28.154.190:35139/ using document root <none> and password file <none>
I20260811 09:12:59.611055 31650 fs_manager.cc:359] Metadata directory not provided
I20260811 09:12:59.611159 31650 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:12:59.611471 31650 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:12:59.614378 31650 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439250192320-29290-0/minicluster-data/master-0/data/instance:
uuid: "259cb751bc0c45ce84292b4483f6d6b3"
format_stamp: "Formatted at 2026-08-11 09:12:59 on dist-test-slave-xt4k"
server_key: "31448cfc769556417c113635df3524ec"
server_key_iv: "83326f8d27b0e7e295c46c39e5a9e45d"
server_key_version: "encryptionkey@0"
I20260811 09:12:59.615190 31650 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439250192320-29290-0/minicluster-data/master-0/wal/instance:
uuid: "259cb751bc0c45ce84292b4483f6d6b3"
format_stamp: "Formatted at 2026-08-11 09:12:59 on dist-test-slave-xt4k"
server_key: "31448cfc769556417c113635df3524ec"
server_key_iv: "83326f8d27b0e7e295c46c39e5a9e45d"
server_key_version: "encryptionkey@0"
I20260811 09:12:59.620572 31650 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260811 09:12:59.624060 31678 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:12:59.625378 31650 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 09:12:59.625581 31650 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439250192320-29290-0/minicluster-data/master-0/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439250192320-29290-0/minicluster-data/master-0/wal
uuid: "259cb751bc0c45ce84292b4483f6d6b3"
format_stamp: "Formatted at 2026-08-11 09:12:59 on dist-test-slave-xt4k"
server_key: "31448cfc769556417c113635df3524ec"
server_key_iv: "83326f8d27b0e7e295c46c39e5a9e45d"
server_key_version: "encryptionkey@0"
I20260811 09:12:59.625799 31650 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439250192320-29290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439250192320-29290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439250192320-29290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:12:59.644968 31650 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:12:59.645834 31650 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:12:59.646106 31650 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:12:59.664176 31650 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.190:37829
I20260811 09:12:59.664218 31793 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.190:37829 every 8 connection(s)
I20260811 09:12:59.665580 31650 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
I20260811 09:12:59.667665 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 31650
I20260811 09:12:59.667850 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439250192320-29290-0/minicluster-data/master-0/wal/instance
I20260811 09:12:59.668454 29290 external_mini_cluster.cc:1428] Setting key 1b6ea6d65cbf7c6b563b1c1ff51f0ec6
I20260811 09:12:59.670130 31794 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:12:59.681833 31798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:12:59.670436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34138 (local address 127.28.154.190:37829)
0811 09:12:59.671417 (+ 981us) server_negotiation.cc:206] Beginning negotiation
0811 09:12:59.671428 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 09:12:59.671469 (+ 41us) server_negotiation.cc:407] Connection header received
0811 09:12:59.673156 (+ 1687us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:12:59.673182 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:12:59.673651 (+ 469us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:12:59.674040 (+ 389us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:12:59.674587 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:59.675991 (+ 1404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:59.677421 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:12:59.678102 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:12:59.678282 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:12:59.679454 (+ 1172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:12:59.679464 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:12:59.679483 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:12:59.680083 (+ 600us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:12:59.680161 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:12:59.680183 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:12:59.680299 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 09:12:59.680556 (+ 257us) server_negotiation.cc:299] Negotiation successful
0811 09:12:59.680867 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":576,"thread_start_us":346,"threads_started":1}
I20260811 09:12:59.682147 31794 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 259cb751bc0c45ce84292b4483f6d6b3: Bootstrap starting.
I20260811 09:12:59.685583 31794 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 259cb751bc0c45ce84292b4483f6d6b3: Neither blocks nor log segments found. Creating new log.
I20260811 09:12:59.686546 31794 log.cc:824] T 00000000000000000000000000000000 P 259cb751bc0c45ce84292b4483f6d6b3: Log is configured to *not* fsync() on all Append() calls
I20260811 09:12:59.689369 31794 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 259cb751bc0c45ce84292b4483f6d6b3: No bootstrap required, opened a new log
I20260811 09:12:59.696943 31794 raft_consensus.cc:348] T 00000000000000000000000000000000 P 259cb751bc0c45ce84292b4483f6d6b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "259cb751bc0c45ce84292b4483f6d6b3" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 37829 } }
I20260811 09:12:59.697364 31794 raft_consensus.cc:374] T 00000000000000000000000000000000 P 259cb751bc0c45ce84292b4483f6d6b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:12:59.697479 31794 raft_consensus.cc:729] T 00000000000000000000000000000000 P 259cb751bc0c45ce84292b4483f6d6b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 259cb751bc0c45ce84292b4483f6d6b3, State: Initialized, Role: FOLLOWER
I20260811 09:12:59.698025 31794 consensus_queue.cc:260] T 00000000000000000000000000000000 P 259cb751bc0c45ce84292b4483f6d6b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "259cb751bc0c45ce84292b4483f6d6b3" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 37829 } }
I20260811 09:12:59.698230 31794 raft_consensus.cc:388] T 00000000000000000000000000000000 P 259cb751bc0c45ce84292b4483f6d6b3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:12:59.698328 31794 raft_consensus.cc:482] T 00000000000000000000000000000000 P 259cb751bc0c45ce84292b4483f6d6b3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:12:59.698453 31794 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 259cb751bc0c45ce84292b4483f6d6b3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:12:59.700634 31794 raft_consensus.cc:504] T 00000000000000000000000000000000 P 259cb751bc0c45ce84292b4483f6d6b3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "259cb751bc0c45ce84292b4483f6d6b3" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 37829 } }
I20260811 09:12:59.701216 31794 leader_election.cc:302] T 00000000000000000000000000000000 P 259cb751bc0c45ce84292b4483f6d6b3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 259cb751bc0c45ce84292b4483f6d6b3; no voters:
I20260811 09:12:59.701845 31794 leader_election.cc:288] T 00000000000000000000000000000000 P 259cb751bc0c45ce84292b4483f6d6b3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:12:59.701941 31803 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 259cb751bc0c45ce84292b4483f6d6b3 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:12:59.702692 31803 raft_consensus.cc:686] T 00000000000000000000000000000000 P 259cb751bc0c45ce84292b4483f6d6b3 [term 1 LEADER]: Becoming Leader. State: Replica: 259cb751bc0c45ce84292b4483f6d6b3, State: Running, Role: LEADER
I20260811 09:12:59.703189 31803 consensus_queue.cc:237] T 00000000000000000000000000000000 P 259cb751bc0c45ce84292b4483f6d6b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "259cb751bc0c45ce84292b4483f6d6b3" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 37829 } }
I20260811 09:12:59.703879 31794 sys_catalog.cc:564] T 00000000000000000000000000000000 P 259cb751bc0c45ce84292b4483f6d6b3 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:12:59.705844 31814 sys_catalog.cc:455] T 00000000000000000000000000000000 P 259cb751bc0c45ce84292b4483f6d6b3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 259cb751bc0c45ce84292b4483f6d6b3. Latest consensus state: current_term: 1 leader_uuid: "259cb751bc0c45ce84292b4483f6d6b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "259cb751bc0c45ce84292b4483f6d6b3" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 37829 } } }
I20260811 09:12:59.706311 31814 sys_catalog.cc:458] T 00000000000000000000000000000000 P 259cb751bc0c45ce84292b4483f6d6b3 [sys.catalog]: This master's current role is: LEADER
I20260811 09:12:59.705870 31812 sys_catalog.cc:455] T 00000000000000000000000000000000 P 259cb751bc0c45ce84292b4483f6d6b3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "259cb751bc0c45ce84292b4483f6d6b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "259cb751bc0c45ce84292b4483f6d6b3" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 37829 } } }
I20260811 09:12:59.706758 31812 sys_catalog.cc:458] T 00000000000000000000000000000000 P 259cb751bc0c45ce84292b4483f6d6b3 [sys.catalog]: This master's current role is: LEADER
I20260811 09:12:59.709772 31820 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:12:59.715076 31820 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:12:59.724359 31820 catalog_manager.cc:1347] Generated new cluster ID: 5a0a726e25914c869edf43a943e5797b
I20260811 09:12:59.724498 31820 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:12:59.736884 31820 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:12:59.738842 31820 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:12:59.752051 31820 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 259cb751bc0c45ce84292b4483f6d6b3: Generated new TSK 0
I20260811 09:12:59.753043 31820 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 09:12:59.768535 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439250192320-29290-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439250192320-29290-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439250192320-29290-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439250192320-29290-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.154.129:0
--local_ip_for_outbound_sockets=127.28.154.129
--webserver_interface=127.28.154.129
--webserver_port=0
--tserver_master_addrs=127.28.154.190:37829
--builtin_ntp_servers=127.28.154.148:34493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 09:13:00.111398 31842 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:13:00.111701 31842 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:13:00.111831 31842 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:13:00.119445 31842 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:13:00.119593 31842 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:13:00.119731 31842 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.154.129
I20260811 09:13:00.129346 31842 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:34493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439250192320-29290-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439250192320-29290-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.154.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439250192320-29290-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.154.129
--webserver_port=0
--tserver_master_addrs=127.28.154.190:37829
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.154.129
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439250192320-29290-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:13:00.130766 31842 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:13:00.132604 31842 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:13:00.146524 31851 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:13:00.146497 31855 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:13:00.146564 31852 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:13:00.147270 31842 server_base.cc:1034] running on GCE node
I20260811 09:13:00.147917 31842 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:13:00.148933 31842 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:13:00.150154 31842 hybrid_clock.cc:648] HybridClock initialized: now 1786439580150099 us; error 51 us; skew 500 ppm
I20260811 09:13:00.150473 31842 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 09:13:00.152727 31842 webserver.cc:460] Webserver started at http://127.28.154.129:32789/ using document root <none> and password file <none>
I20260811 09:13:00.153424 31842 fs_manager.cc:359] Metadata directory not provided
I20260811 09:13:00.153527 31842 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:13:00.153832 31842 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:13:00.156764 31842 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439250192320-29290-0/minicluster-data/ts-0/data/instance:
uuid: "b24979b66ca24c3d90a6465063f3a819"
format_stamp: "Formatted at 2026-08-11 09:13:00 on dist-test-slave-xt4k"
server_key: "f87df2933e50369376cd3afd1cb804dd"
server_key_iv: "b6d9e074f8b32bf6019667b96f0605b4"
server_key_version: "encryptionkey@0"
I20260811 09:13:00.157541 31842 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439250192320-29290-0/minicluster-data/ts-0/wal/instance:
uuid: "b24979b66ca24c3d90a6465063f3a819"
format_stamp: "Formatted at 2026-08-11 09:13:00 on dist-test-slave-xt4k"
server_key: "f87df2933e50369376cd3afd1cb804dd"
server_key_iv: "b6d9e074f8b32bf6019667b96f0605b4"
server_key_version: "encryptionkey@0"
I20260811 09:13:00.163156 31842 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 09:13:00.166807 31863 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:13:00.168282 31842 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 09:13:00.168493 31842 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439250192320-29290-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439250192320-29290-0/minicluster-data/ts-0/wal
uuid: "b24979b66ca24c3d90a6465063f3a819"
format_stamp: "Formatted at 2026-08-11 09:13:00 on dist-test-slave-xt4k"
server_key: "f87df2933e50369376cd3afd1cb804dd"
server_key_iv: "b6d9e074f8b32bf6019667b96f0605b4"
server_key_version: "encryptionkey@0"
I20260811 09:13:00.168709 31842 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439250192320-29290-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439250192320-29290-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439250192320-29290-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:13:00.188966 31842 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:13:00.189883 31842 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:13:00.190197 31842 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:13:00.191256 31842 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:13:00.193084 31842 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:13:00.193189 31842 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:13:00.193310 31842 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:13:00.193382 31842 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:13:00.226117 31842 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.129:38501
I20260811 09:13:00.226168 32066 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.129:38501 every 8 connection(s)
I20260811 09:13:00.227460 31842 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439250192320-29290-0/minicluster-data/ts-0/data/info.pb
I20260811 09:13:00.231889 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 31842
I20260811 09:13:00.232077 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439250192320-29290-0/minicluster-data/ts-0/wal/instance
I20260811 09:13:00.232650 29290 external_mini_cluster.cc:1428] Setting key d257d8b9147a1cb95ce710d736922ef7
I20260811 09:13:00.235848 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439250192320-29290-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439250192320-29290-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439250192320-29290-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439250192320-29290-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.154.130:0
--local_ip_for_outbound_sockets=127.28.154.130
--webserver_interface=127.28.154.130
--webserver_port=0
--tserver_master_addrs=127.28.154.190:37829
--builtin_ntp_servers=127.28.154.148:34493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 09:13:00.241325 32071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:00.230018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.129:43507 (local address 127.28.154.190:37829)
0811 09:13:00.230505 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:00.230511 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:00.231668 (+ 1157us) server_negotiation.cc:407] Connection header received
0811 09:13:00.233951 (+ 2283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:00.233956 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:00.234023 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:00.234117 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:00.235445 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:00.236264 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:00.238477 (+ 2213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:00.238919 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:00.239016 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:00.240585 (+ 1569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:13:00.240591 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:13:00.240594 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:13:00.240615 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:13:00.240660 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:13:00.240672 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:13:00.240744 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:00.240974 (+ 230us) server_negotiation.cc:299] Negotiation successful
0811 09:13:00.241064 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":347,"thread_start_us":240,"threads_started":1}
I20260811 09:13:00.242700 32072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:00.230554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:37829 (local address 127.28.154.129:43507)
0811 09:13:00.231472 (+ 918us) negotiation.cc:107] Waiting for socket to connect
0811 09:13:00.231531 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 09:13:00.233680 (+ 2149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:13:00.234374 (+ 694us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:13:00.234384 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:13:00.234503 (+ 119us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:13:00.235253 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:13:00.235283 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:13:00.236438 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:13:00.236443 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:13:00.238316 (+ 1873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:13:00.238333 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:13:00.239149 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:13:00.239177 (+ 28us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:13:00.239449 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:00.240183 (+ 734us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 09:13:00.240186 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:13:00.240239 (+ 53us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 09:13:00.240383 (+ 144us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:13:00.240763 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:13:00.240773 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:13:00.240781 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 09:13:00.240956 (+ 175us) client_negotiation.cc:240] Negotiation successful
0811 09:13:00.241376 (+ 420us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":267,"threads_started":1}
I20260811 09:13:00.244244 32067 heartbeater.cc:347] Connected to a master server at 127.28.154.190:37829
I20260811 09:13:00.244606 32067 heartbeater.cc:464] Registering TS with master...
I20260811 09:13:00.245520 32067 heartbeater.cc:511] Master 127.28.154.190:37829 requested a full tablet report, sending...
I20260811 09:13:00.247713 31715 ts_manager.cc:194] Registered new tserver with Master: b24979b66ca24c3d90a6465063f3a819 (127.28.154.129:38501)
I20260811 09:13:00.249626 31715 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.28.154.129:43507
W20260811 09:13:00.573185 32073 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:13:00.573482 32073 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:13:00.573583 32073 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:13:00.581473 32073 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:13:00.581590 32073 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:13:00.581699 32073 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.154.130
I20260811 09:13:00.591111 32073 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:34493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439250192320-29290-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439250192320-29290-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.154.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439250192320-29290-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.154.130
--webserver_port=0
--tserver_master_addrs=127.28.154.190:37829
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.154.130
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439250192320-29290-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:13:00.592468 32073 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:13:00.594269 32073 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:13:00.607386 32085 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:13:00.607311 32089 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:13:00.607348 32096 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:13:00.608276 32073 server_base.cc:1034] running on GCE node
I20260811 09:13:00.608940 32073 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:13:00.609922 32073 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:13:00.611164 32073 hybrid_clock.cc:648] HybridClock initialized: now 1786439580611105 us; error 47 us; skew 500 ppm
I20260811 09:13:00.611495 32073 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 09:13:00.613776 32073 webserver.cc:460] Webserver started at http://127.28.154.130:33279/ using document root <none> and password file <none>
I20260811 09:13:00.614466 32073 fs_manager.cc:359] Metadata directory not provided
I20260811 09:13:00.614559 32073 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:13:00.614864 32073 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:13:00.617816 32073 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439250192320-29290-0/minicluster-data/ts-1/data/instance:
uuid: "c34ea2c83a3c4e85ad8d46ec3cab0d67"
format_stamp: "Formatted at 2026-08-11 09:13:00 on dist-test-slave-xt4k"
server_key: "466cbd46714ffcf7b49eb3fbc454cec8"
server_key_iv: "7d36b8e4ac832ffca95f3c1ac96d2ca2"
server_key_version: "encryptionkey@0"
I20260811 09:13:00.618578 32073 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439250192320-29290-0/minicluster-data/ts-1/wal/instance:
uuid: "c34ea2c83a3c4e85ad8d46ec3cab0d67"
format_stamp: "Formatted at 2026-08-11 09:13:00 on dist-test-slave-xt4k"
server_key: "466cbd46714ffcf7b49eb3fbc454cec8"
server_key_iv: "7d36b8e4ac832ffca95f3c1ac96d2ca2"
server_key_version: "encryptionkey@0"
I20260811 09:13:00.623593 32073 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260811 09:13:00.627177 32105 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:13:00.628583 32073 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 09:13:00.628764 32073 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439250192320-29290-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439250192320-29290-0/minicluster-data/ts-1/wal
uuid: "c34ea2c83a3c4e85ad8d46ec3cab0d67"
format_stamp: "Formatted at 2026-08-11 09:13:00 on dist-test-slave-xt4k"
server_key: "466cbd46714ffcf7b49eb3fbc454cec8"
server_key_iv: "7d36b8e4ac832ffca95f3c1ac96d2ca2"
server_key_version: "encryptionkey@0"
I20260811 09:13:00.628991 32073 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439250192320-29290-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439250192320-29290-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439250192320-29290-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:13:00.672542 32073 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:13:00.673452 32073 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:13:00.673797 32073 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:13:00.674808 32073 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:13:00.676440 32073 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:13:00.676528 32073 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:13:00.676651 32073 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:13:00.676718 32073 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:13:00.707484 32073 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.130:34115
I20260811 09:13:00.707536 32306 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.130:34115 every 8 connection(s)
I20260811 09:13:00.708810 32073 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439250192320-29290-0/minicluster-data/ts-1/data/info.pb
I20260811 09:13:00.719091 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 32073
I20260811 09:13:00.719249 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439250192320-29290-0/minicluster-data/ts-1/wal/instance
I20260811 09:13:00.719729 29290 external_mini_cluster.cc:1428] Setting key 6c46976c5b65d6dd9eb499d1ee7ee4e2
I20260811 09:13:00.721043 32071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:00.711436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.130:42505 (local address 127.28.154.190:37829)
0811 09:13:00.711647 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:00.711654 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:00.713021 (+ 1367us) server_negotiation.cc:407] Connection header received
0811 09:13:00.714795 (+ 1774us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:00.714799 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:00.714871 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:00.714959 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:00.716012 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:00.716788 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:00.718317 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:00.718754 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:00.718872 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:00.720271 (+ 1399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:13:00.720278 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:13:00.720282 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:13:00.720306 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:13:00.720365 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:13:00.720384 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:13:00.720471 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:00.720715 (+ 244us) server_negotiation.cc:299] Negotiation successful
0811 09:13:00.720804 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 09:13:00.722298 32311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:00.711953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:37829 (local address 127.28.154.130:42505)
0811 09:13:00.712877 (+ 924us) negotiation.cc:107] Waiting for socket to connect
0811 09:13:00.712912 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0811 09:13:00.714522 (+ 1610us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:13:00.715148 (+ 626us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:13:00.715156 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:13:00.715245 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:13:00.715850 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:13:00.715867 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:13:00.716936 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:13:00.716939 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:13:00.718178 (+ 1239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:13:00.718192 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:13:00.718980 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:13:00.718988 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:13:00.719244 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:00.719962 (+ 718us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 09:13:00.719965 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:13:00.720004 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 09:13:00.720084 (+ 80us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:13:00.720502 (+ 418us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:13:00.720512 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:13:00.720518 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 09:13:00.720676 (+ 158us) client_negotiation.cc:240] Negotiation successful
0811 09:13:00.721040 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":296,"threads_started":1}
I20260811 09:13:00.722791 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439250192320-29290-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439250192320-29290-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439250192320-29290-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439250192320-29290-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.154.131:0
--local_ip_for_outbound_sockets=127.28.154.131
--webserver_interface=127.28.154.131
--webserver_port=0
--tserver_master_addrs=127.28.154.190:37829
--builtin_ntp_servers=127.28.154.148:34493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 09:13:00.723696 32308 heartbeater.cc:347] Connected to a master server at 127.28.154.190:37829
I20260811 09:13:00.723989 32308 heartbeater.cc:464] Registering TS with master...
I20260811 09:13:00.724761 32308 heartbeater.cc:511] Master 127.28.154.190:37829 requested a full tablet report, sending...
I20260811 09:13:00.726068 31715 ts_manager.cc:194] Registered new tserver with Master: c34ea2c83a3c4e85ad8d46ec3cab0d67 (127.28.154.130:34115)
I20260811 09:13:00.726869 31715 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.28.154.130:42505
W20260811 09:13:01.062683 32314 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:13:01.063098 32314 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:13:01.063217 32314 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:13:01.070493 32314 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:13:01.070603 32314 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:13:01.070740 32314 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.154.131
I20260811 09:13:01.080058 32314 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:34493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439250192320-29290-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439250192320-29290-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.154.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439250192320-29290-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.154.131
--webserver_port=0
--tserver_master_addrs=127.28.154.190:37829
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.154.131
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439250192320-29290-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:13:01.081421 32314 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:13:01.083384 32314 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:13:01.097728 32322 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:13:01.097901 32323 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:13:01.098114 32327 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:13:01.099680 32314 server_base.cc:1034] running on GCE node
I20260811 09:13:01.100445 32314 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:13:01.101550 32314 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:13:01.102808 32314 hybrid_clock.cc:648] HybridClock initialized: now 1786439581102750 us; error 51 us; skew 500 ppm
I20260811 09:13:01.103179 32314 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 09:13:01.105624 32314 webserver.cc:460] Webserver started at http://127.28.154.131:42061/ using document root <none> and password file <none>
I20260811 09:13:01.106346 32314 fs_manager.cc:359] Metadata directory not provided
I20260811 09:13:01.106451 32314 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:13:01.106766 32314 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:13:01.110488 32314 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439250192320-29290-0/minicluster-data/ts-2/data/instance:
uuid: "8e6e37d4d45f407db47514fb578c0b35"
format_stamp: "Formatted at 2026-08-11 09:13:01 on dist-test-slave-xt4k"
server_key: "9611c1a1707dd36957014e6ffbc6f820"
server_key_iv: "ddef2cceb156f90a6b5f908d909eab5b"
server_key_version: "encryptionkey@0"
I20260811 09:13:01.111404 32314 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439250192320-29290-0/minicluster-data/ts-2/wal/instance:
uuid: "8e6e37d4d45f407db47514fb578c0b35"
format_stamp: "Formatted at 2026-08-11 09:13:01 on dist-test-slave-xt4k"
server_key: "9611c1a1707dd36957014e6ffbc6f820"
server_key_iv: "ddef2cceb156f90a6b5f908d909eab5b"
server_key_version: "encryptionkey@0"
I20260811 09:13:01.116997 32314 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 09:13:01.120688 32342 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:13:01.122079 32314 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 09:13:01.122238 32314 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439250192320-29290-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439250192320-29290-0/minicluster-data/ts-2/wal
uuid: "8e6e37d4d45f407db47514fb578c0b35"
format_stamp: "Formatted at 2026-08-11 09:13:01 on dist-test-slave-xt4k"
server_key: "9611c1a1707dd36957014e6ffbc6f820"
server_key_iv: "ddef2cceb156f90a6b5f908d909eab5b"
server_key_version: "encryptionkey@0"
I20260811 09:13:01.122460 32314 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439250192320-29290-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439250192320-29290-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439250192320-29290-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:13:01.154069 32314 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:13:01.154949 32314 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:13:01.155239 32314 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:13:01.156272 32314 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:13:01.157855 32314 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:13:01.157945 32314 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:13:01.158041 32314 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:13:01.158109 32314 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:13:01.189971 32314 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.131:38253
I20260811 09:13:01.190073 32551 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.131:38253 every 8 connection(s)
I20260811 09:13:01.191339 32314 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439250192320-29290-0/minicluster-data/ts-2/data/info.pb
I20260811 09:13:01.193897 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 32314
I20260811 09:13:01.194069 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439250192320-29290-0/minicluster-data/ts-2/wal/instance
I20260811 09:13:01.194595 29290 external_mini_cluster.cc:1428] Setting key bc3beb8b5a57f9437d2b6445d1ecd20a
I20260811 09:13:01.205865 32071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:01.194254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.131:56545 (local address 127.28.154.190:37829)
0811 09:13:01.194482 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:01.194489 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:01.196181 (+ 1692us) server_negotiation.cc:407] Connection header received
0811 09:13:01.198354 (+ 2173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:01.198359 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:01.198429 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:01.198543 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:01.199743 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:01.200611 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:01.202328 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:01.202938 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:01.203056 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:01.204506 (+ 1450us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:13:01.204512 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:13:01.204515 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:13:01.204536 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:13:01.204570 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:13:01.204583 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:13:01.204660 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:01.205454 (+ 794us) server_negotiation.cc:299] Negotiation successful
0811 09:13:01.205598 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 09:13:01.206429 32557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:01.194824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:37829 (local address 127.28.154.131:56545)
0811 09:13:01.196014 (+ 1190us) negotiation.cc:107] Waiting for socket to connect
0811 09:13:01.196059 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 09:13:01.198082 (+ 2023us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:13:01.198825 (+ 743us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:13:01.198835 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:13:01.198935 (+ 100us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:13:01.199555 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:13:01.199573 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:13:01.200772 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:13:01.200776 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:13:01.202114 (+ 1338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:13:01.202128 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:13:01.203162 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:13:01.203167 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:13:01.203384 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:01.204144 (+ 760us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 09:13:01.204147 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:13:01.204191 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 09:13:01.204278 (+ 87us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:13:01.204683 (+ 405us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:13:01.204692 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:13:01.204699 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 09:13:01.204890 (+ 191us) client_negotiation.cc:240] Negotiation successful
0811 09:13:01.205245 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":630,"thread_start_us":385,"threads_started":1}
I20260811 09:13:01.208201 32552 heartbeater.cc:347] Connected to a master server at 127.28.154.190:37829
I20260811 09:13:01.208559 32552 heartbeater.cc:464] Registering TS with master...
I20260811 09:13:01.209585 32552 heartbeater.cc:511] Master 127.28.154.190:37829 requested a full tablet report, sending...
I20260811 09:13:01.211319 31715 ts_manager.cc:194] Registered new tserver with Master: 8e6e37d4d45f407db47514fb578c0b35 (127.28.154.131:38253)
I20260811 09:13:01.212147 31715 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.28.154.131:56545
I20260811 09:13:01.223129 29290 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 09:13:01.233804 32071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:01.227523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34140 (local address 127.28.154.190:37829)
0811 09:13:01.227786 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:01.227792 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:01.228089 (+ 297us) server_negotiation.cc:407] Connection header received
0811 09:13:01.228218 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:01.228224 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:01.228317 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:01.228449 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:01.228952 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:01.229824 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:01.231203 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:01.231630 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:01.231757 (+ 127us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:01.232993 (+ 1236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:13:01.232998 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:13:01.233002 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:13:01.233022 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:13:01.233057 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:13:01.233070 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:13:01.233198 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:01.233416 (+ 218us) server_negotiation.cc:299] Negotiation successful
0811 09:13:01.233475 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107}
I20260811 09:13:01.237926 31715 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34140:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 09:13:01.245805 31715 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 09:13:01.253206 32067 heartbeater.cc:503] Master 127.28.154.190:37829 was elected leader, sending a full tablet report...
I20260811 09:13:01.283526 32579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:01.260252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.130:34115 (local address 127.0.0.1:54698)
0811 09:13:01.271950 (+ 11698us) negotiation.cc:107] Waiting for socket to connect
0811 09:13:01.271977 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0811 09:13:01.272116 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:13:01.276249 (+ 4133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:13:01.276253 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:13:01.276280 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:13:01.276661 (+ 381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:13:01.276676 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:13:01.278465 (+ 1789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:13:01.278470 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:13:01.280160 (+ 1690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:13:01.280175 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:13:01.282608 (+ 2433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:13:01.282614 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:13:01.282760 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:01.282802 (+ 42us) client_negotiation.cc:769] Sending connection context
0811 09:13:01.282905 (+ 103us) client_negotiation.cc:240] Negotiation successful
0811 09:13:01.283095 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11553,"mutex_wait_us":1,"thread_start_us":13615,"threads_started":1}
I20260811 09:13:01.283619 32590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:01.262096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35946 (local address 127.28.154.129:38501)
0811 09:13:01.270632 (+ 8536us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:01.270642 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:01.274209 (+ 3567us) server_negotiation.cc:407] Connection header received
0811 09:13:01.274420 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:01.274429 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:01.274739 (+ 310us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:01.274962 (+ 223us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:13:01.276096 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:01.277544 (+ 1448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:01.280766 (+ 3222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:01.282174 (+ 1408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:01.282350 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:01.282556 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:01.283087 (+ 531us) server_negotiation.cc:299] Negotiation successful
0811 09:13:01.283265 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8340,"thread_start_us":8192,"threads_started":1}
I20260811 09:13:01.283699 32585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:01.260543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54698 (local address 127.28.154.130:34115)
0811 09:13:01.275647 (+ 15104us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:01.275655 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:01.275683 (+ 28us) server_negotiation.cc:407] Connection header received
0811 09:13:01.275785 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:01.275792 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:01.275951 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:01.276147 (+ 196us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:13:01.276853 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:01.278321 (+ 1468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:01.280374 (+ 2053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:01.281964 (+ 1590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:01.282138 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:01.282392 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:01.283206 (+ 814us) server_negotiation.cc:299] Negotiation successful
0811 09:13:01.283375 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14973,"thread_start_us":14862,"threads_started":1}
I20260811 09:13:01.284157 32589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:01.260065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.131:38253 (local address 127.0.0.1:37398)
0811 09:13:01.269712 (+ 9647us) negotiation.cc:107] Waiting for socket to connect
0811 09:13:01.269778 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0811 09:13:01.270002 (+ 224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:13:01.275619 (+ 5617us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:13:01.275624 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:13:01.275654 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:13:01.276089 (+ 435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:13:01.276102 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:13:01.277760 (+ 1658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:13:01.277765 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:13:01.280497 (+ 2732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:13:01.280519 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:13:01.282560 (+ 2041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:13:01.282567 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:13:01.282827 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:01.282891 (+ 64us) client_negotiation.cc:769] Sending connection context
0811 09:13:01.282979 (+ 88us) client_negotiation.cc:240] Negotiation successful
0811 09:13:01.283143 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9325,"mutex_wait_us":170,"thread_start_us":12130,"threads_started":1}
I20260811 09:13:01.284152 32586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:01.260439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37398 (local address 127.28.154.131:38253)
0811 09:13:01.274775 (+ 14336us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:01.274783 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:01.274816 (+ 33us) server_negotiation.cc:407] Connection header received
0811 09:13:01.274910 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:01.274917 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:01.275149 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:01.275337 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:13:01.276251 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:01.277607 (+ 1356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:01.280756 (+ 3149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:01.281956 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:01.282148 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:01.282393 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:01.283539 (+ 1146us) server_negotiation.cc:299] Negotiation successful
0811 09:13:01.283701 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14157,"thread_start_us":15187,"threads_started":1}
I20260811 09:13:01.285742 32584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:01.261422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.129:38501 (local address 127.0.0.1:35946)
0811 09:13:01.274077 (+ 12655us) negotiation.cc:107] Waiting for socket to connect
0811 09:13:01.274096 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 09:13:01.274223 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:13:01.275268 (+ 1045us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:13:01.275276 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:13:01.275317 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:13:01.275876 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:13:01.275890 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:13:01.277719 (+ 1829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:13:01.277724 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:13:01.280581 (+ 2857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:13:01.280591 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:13:01.282646 (+ 2055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:13:01.282654 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:13:01.282893 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:01.282920 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 09:13:01.282995 (+ 75us) client_negotiation.cc:240] Negotiation successful
0811 09:13:01.285464 (+ 2469us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12543,"mutex_wait_us":2510,"thread_start_us":7805,"threads_started":1}
I20260811 09:13:01.288466 32184 tablet_service.cc:1467] Processing CreateTablet for tablet b1db1834fcb64d60a773d4055c0df144 (DEFAULT_TABLE table=test-table [id=4d18a8fc551d445fa1f9cb5a49abf2d5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:13:01.288475 32433 tablet_service.cc:1467] Processing CreateTablet for tablet b1db1834fcb64d60a773d4055c0df144 (DEFAULT_TABLE table=test-table [id=4d18a8fc551d445fa1f9cb5a49abf2d5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:13:01.288856 31950 tablet_service.cc:1467] Processing CreateTablet for tablet b1db1834fcb64d60a773d4055c0df144 (DEFAULT_TABLE table=test-table [id=4d18a8fc551d445fa1f9cb5a49abf2d5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:13:01.290014 31950 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b1db1834fcb64d60a773d4055c0df144. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:13:01.290016 32433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b1db1834fcb64d60a773d4055c0df144. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:13:01.290030 32184 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b1db1834fcb64d60a773d4055c0df144. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:13:01.302054 32591 tablet_bootstrap.cc:492] T b1db1834fcb64d60a773d4055c0df144 P b24979b66ca24c3d90a6465063f3a819: Bootstrap starting.
I20260811 09:13:01.304628 32595 tablet_bootstrap.cc:492] T b1db1834fcb64d60a773d4055c0df144 P c34ea2c83a3c4e85ad8d46ec3cab0d67: Bootstrap starting.
I20260811 09:13:01.305480 32596 tablet_bootstrap.cc:492] T b1db1834fcb64d60a773d4055c0df144 P 8e6e37d4d45f407db47514fb578c0b35: Bootstrap starting.
I20260811 09:13:01.307693 32591 tablet_bootstrap.cc:654] T b1db1834fcb64d60a773d4055c0df144 P b24979b66ca24c3d90a6465063f3a819: Neither blocks nor log segments found. Creating new log.
I20260811 09:13:01.309015 32596 tablet_bootstrap.cc:654] T b1db1834fcb64d60a773d4055c0df144 P 8e6e37d4d45f407db47514fb578c0b35: Neither blocks nor log segments found. Creating new log.
I20260811 09:13:01.309027 32591 log.cc:824] T b1db1834fcb64d60a773d4055c0df144 P b24979b66ca24c3d90a6465063f3a819: Log is configured to *not* fsync() on all Append() calls
I20260811 09:13:01.310242 32596 log.cc:824] T b1db1834fcb64d60a773d4055c0df144 P 8e6e37d4d45f407db47514fb578c0b35: Log is configured to *not* fsync() on all Append() calls
I20260811 09:13:01.310860 32595 tablet_bootstrap.cc:654] T b1db1834fcb64d60a773d4055c0df144 P c34ea2c83a3c4e85ad8d46ec3cab0d67: Neither blocks nor log segments found. Creating new log.
I20260811 09:13:01.311975 32595 log.cc:824] T b1db1834fcb64d60a773d4055c0df144 P c34ea2c83a3c4e85ad8d46ec3cab0d67: Log is configured to *not* fsync() on all Append() calls
I20260811 09:13:01.312407 32591 tablet_bootstrap.cc:492] T b1db1834fcb64d60a773d4055c0df144 P b24979b66ca24c3d90a6465063f3a819: No bootstrap required, opened a new log
I20260811 09:13:01.312755 32591 ts_tablet_manager.cc:1397] T b1db1834fcb64d60a773d4055c0df144 P b24979b66ca24c3d90a6465063f3a819: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260811 09:13:01.313280 32596 tablet_bootstrap.cc:492] T b1db1834fcb64d60a773d4055c0df144 P 8e6e37d4d45f407db47514fb578c0b35: No bootstrap required, opened a new log
I20260811 09:13:01.313535 32596 ts_tablet_manager.cc:1397] T b1db1834fcb64d60a773d4055c0df144 P 8e6e37d4d45f407db47514fb578c0b35: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260811 09:13:01.314590 32595 tablet_bootstrap.cc:492] T b1db1834fcb64d60a773d4055c0df144 P c34ea2c83a3c4e85ad8d46ec3cab0d67: No bootstrap required, opened a new log
I20260811 09:13:01.314889 32595 ts_tablet_manager.cc:1397] T b1db1834fcb64d60a773d4055c0df144 P c34ea2c83a3c4e85ad8d46ec3cab0d67: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.002s
I20260811 09:13:01.321552 32595 raft_consensus.cc:348] T b1db1834fcb64d60a773d4055c0df144 P c34ea2c83a3c4e85ad8d46ec3cab0d67 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e6e37d4d45f407db47514fb578c0b35" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 38253 } } peers { permanent_uuid: "c34ea2c83a3c4e85ad8d46ec3cab0d67" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34115 } } peers { permanent_uuid: "b24979b66ca24c3d90a6465063f3a819" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 38501 } }
I20260811 09:13:01.321913 32595 raft_consensus.cc:374] T b1db1834fcb64d60a773d4055c0df144 P c34ea2c83a3c4e85ad8d46ec3cab0d67 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:13:01.322053 32595 raft_consensus.cc:729] T b1db1834fcb64d60a773d4055c0df144 P c34ea2c83a3c4e85ad8d46ec3cab0d67 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c34ea2c83a3c4e85ad8d46ec3cab0d67, State: Initialized, Role: FOLLOWER
I20260811 09:13:01.322657 32595 consensus_queue.cc:260] T b1db1834fcb64d60a773d4055c0df144 P c34ea2c83a3c4e85ad8d46ec3cab0d67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e6e37d4d45f407db47514fb578c0b35" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 38253 } } peers { permanent_uuid: "c34ea2c83a3c4e85ad8d46ec3cab0d67" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34115 } } peers { permanent_uuid: "b24979b66ca24c3d90a6465063f3a819" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 38501 } }
I20260811 09:13:01.323036 32591 raft_consensus.cc:348] T b1db1834fcb64d60a773d4055c0df144 P b24979b66ca24c3d90a6465063f3a819 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e6e37d4d45f407db47514fb578c0b35" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 38253 } } peers { permanent_uuid: "c34ea2c83a3c4e85ad8d46ec3cab0d67" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34115 } } peers { permanent_uuid: "b24979b66ca24c3d90a6465063f3a819" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 38501 } }
I20260811 09:13:01.323531 32591 raft_consensus.cc:374] T b1db1834fcb64d60a773d4055c0df144 P b24979b66ca24c3d90a6465063f3a819 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:13:01.323693 32308 heartbeater.cc:503] Master 127.28.154.190:37829 was elected leader, sending a full tablet report...
I20260811 09:13:01.323674 32591 raft_consensus.cc:729] T b1db1834fcb64d60a773d4055c0df144 P b24979b66ca24c3d90a6465063f3a819 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b24979b66ca24c3d90a6465063f3a819, State: Initialized, Role: FOLLOWER
I20260811 09:13:01.323674 32596 raft_consensus.cc:348] T b1db1834fcb64d60a773d4055c0df144 P 8e6e37d4d45f407db47514fb578c0b35 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e6e37d4d45f407db47514fb578c0b35" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 38253 } } peers { permanent_uuid: "c34ea2c83a3c4e85ad8d46ec3cab0d67" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34115 } } peers { permanent_uuid: "b24979b66ca24c3d90a6465063f3a819" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 38501 } }
I20260811 09:13:01.324077 32596 raft_consensus.cc:374] T b1db1834fcb64d60a773d4055c0df144 P 8e6e37d4d45f407db47514fb578c0b35 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:13:01.324168 32596 raft_consensus.cc:729] T b1db1834fcb64d60a773d4055c0df144 P 8e6e37d4d45f407db47514fb578c0b35 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e6e37d4d45f407db47514fb578c0b35, State: Initialized, Role: FOLLOWER
I20260811 09:13:01.324398 32591 consensus_queue.cc:260] T b1db1834fcb64d60a773d4055c0df144 P b24979b66ca24c3d90a6465063f3a819 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e6e37d4d45f407db47514fb578c0b35" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 38253 } } peers { permanent_uuid: "c34ea2c83a3c4e85ad8d46ec3cab0d67" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34115 } } peers { permanent_uuid: "b24979b66ca24c3d90a6465063f3a819" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 38501 } }
I20260811 09:13:01.324887 32596 consensus_queue.cc:260] T b1db1834fcb64d60a773d4055c0df144 P 8e6e37d4d45f407db47514fb578c0b35 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e6e37d4d45f407db47514fb578c0b35" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 38253 } } peers { permanent_uuid: "c34ea2c83a3c4e85ad8d46ec3cab0d67" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34115 } } peers { permanent_uuid: "b24979b66ca24c3d90a6465063f3a819" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 38501 } }
I20260811 09:13:01.326188 32595 ts_tablet_manager.cc:1428] T b1db1834fcb64d60a773d4055c0df144 P c34ea2c83a3c4e85ad8d46ec3cab0d67: Time spent starting tablet: real 0.011s user 0.003s sys 0.007s
I20260811 09:13:01.327224 32552 heartbeater.cc:503] Master 127.28.154.190:37829 was elected leader, sending a full tablet report...
I20260811 09:13:01.328840 32596 ts_tablet_manager.cc:1428] T b1db1834fcb64d60a773d4055c0df144 P 8e6e37d4d45f407db47514fb578c0b35: Time spent starting tablet: real 0.015s user 0.014s sys 0.000s
I20260811 09:13:01.332027 32591 ts_tablet_manager.cc:1428] T b1db1834fcb64d60a773d4055c0df144 P b24979b66ca24c3d90a6465063f3a819: Time spent starting tablet: real 0.019s user 0.018s sys 0.000s
W20260811 09:13:01.443276 32556 tablet.cc:2369] T b1db1834fcb64d60a773d4055c0df144 P 8e6e37d4d45f407db47514fb578c0b35: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 09:13:01.460887 32310 tablet.cc:2369] T b1db1834fcb64d60a773d4055c0df144 P c34ea2c83a3c4e85ad8d46ec3cab0d67: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 09:13:01.479947 32070 tablet.cc:2369] T b1db1834fcb64d60a773d4055c0df144 P b24979b66ca24c3d90a6465063f3a819: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 09:13:01.481701 32613 raft_consensus.cc:482] T b1db1834fcb64d60a773d4055c0df144 P 8e6e37d4d45f407db47514fb578c0b35 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:13:01.481983 32613 raft_consensus.cc:504] T b1db1834fcb64d60a773d4055c0df144 P 8e6e37d4d45f407db47514fb578c0b35 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e6e37d4d45f407db47514fb578c0b35" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 38253 } } peers { permanent_uuid: "c34ea2c83a3c4e85ad8d46ec3cab0d67" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34115 } } peers { permanent_uuid: "b24979b66ca24c3d90a6465063f3a819" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 38501 } }
I20260811 09:13:01.483698 32613 leader_election.cc:288] T b1db1834fcb64d60a773d4055c0df144 P 8e6e37d4d45f407db47514fb578c0b35 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c34ea2c83a3c4e85ad8d46ec3cab0d67 (127.28.154.130:34115), b24979b66ca24c3d90a6465063f3a819 (127.28.154.129:38501)
I20260811 09:13:01.490321 32585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:01.484275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.131:38509 (local address 127.28.154.130:34115)
0811 09:13:01.484531 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:01.484537 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:01.484557 (+ 20us) server_negotiation.cc:407] Connection header received
0811 09:13:01.484634 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:01.484639 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:01.484729 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:01.484822 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:13:01.485279 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:01.486206 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:01.488498 (+ 2292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:01.489505 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:01.489628 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:01.489677 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:01.490013 (+ 336us) server_negotiation.cc:299] Negotiation successful
0811 09:13:01.490102 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122}
I20260811 09:13:01.490419 32557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:01.483973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.130:34115 (local address 127.28.154.131:38509)
0811 09:13:01.484157 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0811 09:13:01.484179 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 09:13:01.484325 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:13:01.484894 (+ 569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:13:01.484898 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:13:01.484927 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:13:01.485120 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:13:01.485132 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:13:01.486331 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:13:01.486337 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:13:01.488314 (+ 1977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:13:01.488326 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:13:01.489656 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:13:01.489662 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:13:01.489868 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:01.489903 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 09:13:01.489998 (+ 95us) client_negotiation.cc:240] Negotiation successful
0811 09:13:01.490144 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260811 09:13:01.491075 32590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:01.484316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.131:59713 (local address 127.28.154.129:38501)
0811 09:13:01.484573 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:01.484580 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:01.485059 (+ 479us) server_negotiation.cc:407] Connection header received
0811 09:13:01.485186 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:01.485191 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:01.485281 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:01.485348 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:13:01.485970 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:01.486885 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:01.489190 (+ 2305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:01.490173 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:01.490319 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:01.490379 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:01.490711 (+ 332us) server_negotiation.cc:299] Negotiation successful
0811 09:13:01.490844 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118}
I20260811 09:13:01.491487 32217 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1db1834fcb64d60a773d4055c0df144" candidate_uuid: "8e6e37d4d45f407db47514fb578c0b35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c34ea2c83a3c4e85ad8d46ec3cab0d67" is_pre_election: true
I20260811 09:13:01.491739 32615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:01.483973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.129:38501 (local address 127.28.154.131:59713)
0811 09:13:01.484955 (+ 982us) negotiation.cc:107] Waiting for socket to connect
0811 09:13:01.484975 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 09:13:01.485077 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:13:01.485398 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:13:01.485410 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:13:01.485435 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:13:01.485824 (+ 389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:13:01.485835 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:13:01.487113 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:13:01.487118 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:13:01.488993 (+ 1875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:13:01.489007 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:13:01.490340 (+ 1333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:13:01.490346 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:13:01.490522 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:01.490557 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 09:13:01.491258 (+ 701us) client_negotiation.cc:240] Negotiation successful
0811 09:13:01.491386 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":850,"mutex_wait_us":263,"thread_start_us":349,"threads_started":1}
I20260811 09:13:01.492098 32217 raft_consensus.cc:2454] T b1db1834fcb64d60a773d4055c0df144 P c34ea2c83a3c4e85ad8d46ec3cab0d67 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e6e37d4d45f407db47514fb578c0b35 in term 0.
I20260811 09:13:01.492799 32344 leader_election.cc:302] T b1db1834fcb64d60a773d4055c0df144 P 8e6e37d4d45f407db47514fb578c0b35 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e6e37d4d45f407db47514fb578c0b35, c34ea2c83a3c4e85ad8d46ec3cab0d67; no voters:
I20260811 09:13:01.492753 31992 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1db1834fcb64d60a773d4055c0df144" candidate_uuid: "8e6e37d4d45f407db47514fb578c0b35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b24979b66ca24c3d90a6465063f3a819" is_pre_election: true
I20260811 09:13:01.493209 32613 raft_consensus.cc:2781] T b1db1834fcb64d60a773d4055c0df144 P 8e6e37d4d45f407db47514fb578c0b35 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 09:13:01.493278 31992 raft_consensus.cc:2454] T b1db1834fcb64d60a773d4055c0df144 P b24979b66ca24c3d90a6465063f3a819 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8e6e37d4d45f407db47514fb578c0b35 in term 0.
I20260811 09:13:01.493340 32613 raft_consensus.cc:482] T b1db1834fcb64d60a773d4055c0df144 P 8e6e37d4d45f407db47514fb578c0b35 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 09:13:01.493439 32613 raft_consensus.cc:3037] T b1db1834fcb64d60a773d4055c0df144 P 8e6e37d4d45f407db47514fb578c0b35 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:13:01.495711 32613 raft_consensus.cc:504] T b1db1834fcb64d60a773d4055c0df144 P 8e6e37d4d45f407db47514fb578c0b35 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e6e37d4d45f407db47514fb578c0b35" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 38253 } } peers { permanent_uuid: "c34ea2c83a3c4e85ad8d46ec3cab0d67" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34115 } } peers { permanent_uuid: "b24979b66ca24c3d90a6465063f3a819" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 38501 } }
I20260811 09:13:01.496271 32613 leader_election.cc:288] T b1db1834fcb64d60a773d4055c0df144 P 8e6e37d4d45f407db47514fb578c0b35 [CANDIDATE]: Term 1 election: Requested vote from peers c34ea2c83a3c4e85ad8d46ec3cab0d67 (127.28.154.130:34115), b24979b66ca24c3d90a6465063f3a819 (127.28.154.129:38501)
I20260811 09:13:01.496750 32217 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1db1834fcb64d60a773d4055c0df144" candidate_uuid: "8e6e37d4d45f407db47514fb578c0b35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c34ea2c83a3c4e85ad8d46ec3cab0d67"
I20260811 09:13:01.496778 31992 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1db1834fcb64d60a773d4055c0df144" candidate_uuid: "8e6e37d4d45f407db47514fb578c0b35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b24979b66ca24c3d90a6465063f3a819"
I20260811 09:13:01.496922 32217 raft_consensus.cc:3037] T b1db1834fcb64d60a773d4055c0df144 P c34ea2c83a3c4e85ad8d46ec3cab0d67 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:13:01.496941 31992 raft_consensus.cc:3037] T b1db1834fcb64d60a773d4055c0df144 P b24979b66ca24c3d90a6465063f3a819 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:13:01.499452 32217 raft_consensus.cc:2454] T b1db1834fcb64d60a773d4055c0df144 P c34ea2c83a3c4e85ad8d46ec3cab0d67 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e6e37d4d45f407db47514fb578c0b35 in term 1.
I20260811 09:13:01.499488 31992 raft_consensus.cc:2454] T b1db1834fcb64d60a773d4055c0df144 P b24979b66ca24c3d90a6465063f3a819 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8e6e37d4d45f407db47514fb578c0b35 in term 1.
I20260811 09:13:01.499819 32344 leader_election.cc:302] T b1db1834fcb64d60a773d4055c0df144 P 8e6e37d4d45f407db47514fb578c0b35 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8e6e37d4d45f407db47514fb578c0b35, c34ea2c83a3c4e85ad8d46ec3cab0d67; no voters:
I20260811 09:13:01.500105 32613 raft_consensus.cc:2781] T b1db1834fcb64d60a773d4055c0df144 P 8e6e37d4d45f407db47514fb578c0b35 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:13:01.500603 32613 raft_consensus.cc:686] T b1db1834fcb64d60a773d4055c0df144 P 8e6e37d4d45f407db47514fb578c0b35 [term 1 LEADER]: Becoming Leader. State: Replica: 8e6e37d4d45f407db47514fb578c0b35, State: Running, Role: LEADER
I20260811 09:13:01.501137 32613 consensus_queue.cc:237] T b1db1834fcb64d60a773d4055c0df144 P 8e6e37d4d45f407db47514fb578c0b35 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e6e37d4d45f407db47514fb578c0b35" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 38253 } } peers { permanent_uuid: "c34ea2c83a3c4e85ad8d46ec3cab0d67" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34115 } } peers { permanent_uuid: "b24979b66ca24c3d90a6465063f3a819" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 38501 } }
I20260811 09:13:01.505865 31714 catalog_manager.cc:5495] T b1db1834fcb64d60a773d4055c0df144 P 8e6e37d4d45f407db47514fb578c0b35 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8e6e37d4d45f407db47514fb578c0b35 (127.28.154.131). New cstate: current_term: 1 leader_uuid: "8e6e37d4d45f407db47514fb578c0b35" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e6e37d4d45f407db47514fb578c0b35" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 38253 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c34ea2c83a3c4e85ad8d46ec3cab0d67" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b24979b66ca24c3d90a6465063f3a819" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 38501 } health_report { overall_health: UNKNOWN } } }
I20260811 09:13:01.525799 32586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:01.520276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37404 (local address 127.28.154.131:38253)
0811 09:13:01.520508 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:01.520514 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:01.520595 (+ 81us) server_negotiation.cc:407] Connection header received
0811 09:13:01.520809 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:01.520813 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:01.520892 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:01.521027 (+ 135us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:13:01.521453 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:01.522195 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:01.523524 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:01.523926 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:01.524044 (+ 118us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:01.524683 (+ 639us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:13:01.525188 (+ 505us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:13:01.525342 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:01.525518 (+ 176us) server_negotiation.cc:299] Negotiation successful
0811 09:13:01.525562 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260811 09:13:01.534118 31992 raft_consensus.cc:1261] T b1db1834fcb64d60a773d4055c0df144 P b24979b66ca24c3d90a6465063f3a819 [term 1 FOLLOWER]: Refusing update from remote peer 8e6e37d4d45f407db47514fb578c0b35: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:13:01.534119 32217 raft_consensus.cc:1261] T b1db1834fcb64d60a773d4055c0df144 P c34ea2c83a3c4e85ad8d46ec3cab0d67 [term 1 FOLLOWER]: Refusing update from remote peer 8e6e37d4d45f407db47514fb578c0b35: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:13:01.535015 32616 consensus_queue.cc:1035] T b1db1834fcb64d60a773d4055c0df144 P 8e6e37d4d45f407db47514fb578c0b35 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c34ea2c83a3c4e85ad8d46ec3cab0d67" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 34115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:13:01.535347 32613 consensus_queue.cc:1035] T b1db1834fcb64d60a773d4055c0df144 P 8e6e37d4d45f407db47514fb578c0b35 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b24979b66ca24c3d90a6465063f3a819" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 38501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:13:01.547629 32628 mvcc.cc:205] Tried to move back new op lower bound from 7317256525953597440 to 7317256525832593408. Current Snapshot: MvccSnapshot[applied={T|T < 7317256525953597440}]
I20260811 09:13:01.547731 32629 mvcc.cc:205] Tried to move back new op lower bound from 7317256525953597440 to 7317256525832593408. Current Snapshot: MvccSnapshot[applied={T|T < 7317256525953597440}]
I20260811 09:13:01.563071 31715 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34140:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:13:01.563356 31715 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34140:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:13:01.567008 31715 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 259cb751bc0c45ce84292b4483f6d6b3: Sending DeleteTablet for 3 replicas of tablet b1db1834fcb64d60a773d4055c0df144
I20260811 09:13:01.568168 31950 tablet_service.cc:1514] Processing DeleteTablet for tablet b1db1834fcb64d60a773d4055c0df144 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:13:01 UTC) from {username='slave'} at 127.0.0.1:35946
I20260811 09:13:01.568185 32433 tablet_service.cc:1514] Processing DeleteTablet for tablet b1db1834fcb64d60a773d4055c0df144 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:13:01 UTC) from {username='slave'} at 127.0.0.1:37398
I20260811 09:13:01.568189 32184 tablet_service.cc:1514] Processing DeleteTablet for tablet b1db1834fcb64d60a773d4055c0df144 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:13:01 UTC) from {username='slave'} at 127.0.0.1:54698
I20260811 09:13:01.569689 32647 tablet_replica.cc:330] T b1db1834fcb64d60a773d4055c0df144 P c34ea2c83a3c4e85ad8d46ec3cab0d67: stopping tablet replica
I20260811 09:13:01.570334 32647 raft_consensus.cc:2229] T b1db1834fcb64d60a773d4055c0df144 P c34ea2c83a3c4e85ad8d46ec3cab0d67 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:13:01.570497 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 31842
I20260811 09:13:01.570962 32647 raft_consensus.cc:2258] T b1db1834fcb64d60a773d4055c0df144 P c34ea2c83a3c4e85ad8d46ec3cab0d67 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:13:01.571796 32650 tablet_replica.cc:330] T b1db1834fcb64d60a773d4055c0df144 P b24979b66ca24c3d90a6465063f3a819: stopping tablet replica
I20260811 09:13:01.572242 32650 raft_consensus.cc:2229] T b1db1834fcb64d60a773d4055c0df144 P b24979b66ca24c3d90a6465063f3a819 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:13:01.572777 32650 raft_consensus.cc:2258] T b1db1834fcb64d60a773d4055c0df144 P b24979b66ca24c3d90a6465063f3a819 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:13:01.573554 32649 tablet_replica.cc:330] T b1db1834fcb64d60a773d4055c0df144 P 8e6e37d4d45f407db47514fb578c0b35: stopping tablet replica
I20260811 09:13:01.573983 32649 raft_consensus.cc:2229] T b1db1834fcb64d60a773d4055c0df144 P 8e6e37d4d45f407db47514fb578c0b35 [term 1 LEADER]: Raft consensus shutting down.
I20260811 09:13:01.574733 32649 raft_consensus.cc:2258] T b1db1834fcb64d60a773d4055c0df144 P 8e6e37d4d45f407db47514fb578c0b35 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:13:01.575580 32647 ts_tablet_manager.cc:1905] T b1db1834fcb64d60a773d4055c0df144 P c34ea2c83a3c4e85ad8d46ec3cab0d67: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:13:01.576805 32650 ts_tablet_manager.cc:1905] T b1db1834fcb64d60a773d4055c0df144 P b24979b66ca24c3d90a6465063f3a819: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:13:01.577381 32649 ts_tablet_manager.cc:1905] T b1db1834fcb64d60a773d4055c0df144 P 8e6e37d4d45f407db47514fb578c0b35: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:13:01.583325 32647 ts_tablet_manager.cc:1918] T b1db1834fcb64d60a773d4055c0df144 P c34ea2c83a3c4e85ad8d46ec3cab0d67: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:13:01.583613 32647 log.cc:1196] T b1db1834fcb64d60a773d4055c0df144 P c34ea2c83a3c4e85ad8d46ec3cab0d67: Deleting WAL directory at /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439250192320-29290-0/minicluster-data/ts-1/wal/wals/b1db1834fcb64d60a773d4055c0df144
I20260811 09:13:01.583766 32650 ts_tablet_manager.cc:1918] T b1db1834fcb64d60a773d4055c0df144 P b24979b66ca24c3d90a6465063f3a819: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:13:01.584035 32650 log.cc:1196] T b1db1834fcb64d60a773d4055c0df144 P b24979b66ca24c3d90a6465063f3a819: Deleting WAL directory at /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439250192320-29290-0/minicluster-data/ts-0/wal/wals/b1db1834fcb64d60a773d4055c0df144
I20260811 09:13:01.584285 32647 ts_tablet_manager.cc:1939] T b1db1834fcb64d60a773d4055c0df144 P c34ea2c83a3c4e85ad8d46ec3cab0d67: Deleting consensus metadata
I20260811 09:13:01.584503 32649 ts_tablet_manager.cc:1918] T b1db1834fcb64d60a773d4055c0df144 P 8e6e37d4d45f407db47514fb578c0b35: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:13:01.584698 32650 ts_tablet_manager.cc:1939] T b1db1834fcb64d60a773d4055c0df144 P b24979b66ca24c3d90a6465063f3a819: Deleting consensus metadata
I20260811 09:13:01.586617 31679 catalog_manager.cc:4841] TS b24979b66ca24c3d90a6465063f3a819 (127.28.154.129:38501): tablet b1db1834fcb64d60a773d4055c0df144 (table test-table [id=4d18a8fc551d445fa1f9cb5a49abf2d5]) successfully deleted
I20260811 09:13:01.587275 31680 catalog_manager.cc:4841] TS c34ea2c83a3c4e85ad8d46ec3cab0d67 (127.28.154.130:34115): tablet b1db1834fcb64d60a773d4055c0df144 (table test-table [id=4d18a8fc551d445fa1f9cb5a49abf2d5]) successfully deleted
I20260811 09:13:01.588639 32649 log.cc:1196] T b1db1834fcb64d60a773d4055c0df144 P 8e6e37d4d45f407db47514fb578c0b35: Deleting WAL directory at /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786439250192320-29290-0/minicluster-data/ts-2/wal/wals/b1db1834fcb64d60a773d4055c0df144
I20260811 09:13:01.589249 32649 ts_tablet_manager.cc:1939] T b1db1834fcb64d60a773d4055c0df144 P 8e6e37d4d45f407db47514fb578c0b35: Deleting consensus metadata
I20260811 09:13:01.590914 31683 catalog_manager.cc:4841] TS 8e6e37d4d45f407db47514fb578c0b35 (127.28.154.131:38253): tablet b1db1834fcb64d60a773d4055c0df144 (table test-table [id=4d18a8fc551d445fa1f9cb5a49abf2d5]) successfully deleted
I20260811 09:13:01.591962 32590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:01.573031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35962 (local address 127.28.154.129:38501)
0811 09:13:01.574270 (+ 1239us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:01.574276 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:01.576078 (+ 1802us) server_negotiation.cc:407] Connection header received
0811 09:13:01.584999 (+ 8921us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:01.585006 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:01.585135 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:01.585220 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:01.585939 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:01.586974 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:01.588528 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:01.589127 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:01.589266 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:01.590505 (+ 1239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:13:01.590517 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:13:01.590535 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:13:01.591280 (+ 745us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:13:01.591335 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:13:01.591356 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:13:01.591442 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:01.591627 (+ 185us) server_negotiation.cc:299] Negotiation successful
0811 09:13:01.591717 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1108}
I20260811 09:13:01.592664 32058 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:13:01.738554 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 31842
I20260811 09:13:01.764619 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 32073
I20260811 09:13:01.772445 32585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:01.765391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54702 (local address 127.28.154.130:34115)
0811 09:13:01.765853 (+ 462us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:01.765858 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:01.765875 (+ 17us) server_negotiation.cc:407] Connection header received
0811 09:13:01.765925 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:01.765929 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:01.766017 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:01.766105 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:01.766613 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:01.767378 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:01.769277 (+ 1899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:01.769701 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:01.769800 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:01.770993 (+ 1193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:13:01.771002 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:13:01.771017 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:13:01.771646 (+ 629us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:13:01.771704 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:13:01.771732 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:13:01.771817 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:01.771960 (+ 143us) server_negotiation.cc:299] Negotiation successful
0811 09:13:01.772071 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345}
I20260811 09:13:01.773072 32294 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:13:01.862149 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 32073
I20260811 09:13:01.891122 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 32314
I20260811 09:13:01.899024 32586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:01.892041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37408 (local address 127.28.154.131:38253)
0811 09:13:01.892193 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:01.892197 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:01.892237 (+ 40us) server_negotiation.cc:407] Connection header received
0811 09:13:01.892309 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:01.892314 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:01.892455 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:01.892560 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:01.893037 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:01.893845 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:01.895697 (+ 1852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:01.896168 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:01.896262 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:01.897545 (+ 1283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:13:01.897555 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:13:01.897572 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:13:01.898200 (+ 628us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:13:01.898254 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:13:01.898287 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:13:01.898371 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:01.898540 (+ 169us) server_negotiation.cc:299] Negotiation successful
0811 09:13:01.898629 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 09:13:01.899839 32545 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:13:02.024236 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 32314
I20260811 09:13:02.047834 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 31650
I20260811 09:13:02.049023 31787 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:13:02.161451 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 31650
2026-08-11T09:13:02Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (3026 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439250192320-29290-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:13:02 dist-test-slave-xt4k krb5kdc[32667](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:13:02 dist-test-slave-xt4k krb5kdc[32667](info): set up 2 sockets
Aug 11 09:13:02 dist-test-slave-xt4k krb5kdc[32667](info): commencing operation
krb5kdc: starting...
W20260811 09:13:03.478195 29290 mini_kdc.cc:121] Time spent starting KDC: real 1.269s user 0.000s sys 0.047s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439250192320-29290-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439250192320-29290-0/krb5kdc/test-user.keytab.
Aug 11 09:13:03 dist-test-slave-xt4k krb5kdc[32667](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439583, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:13:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:13:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.154.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.154.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.154.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab.
Entry for principal kudu/127.28.154.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab.
I20260811 09:13:03.792610 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439250192320-29290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439250192320-29290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439250192320-29290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.154.190:34389
--webserver_interface=127.28.154.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab
--principal=kudu/127.28.154.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:34169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.154.190:34389
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:13:04.134876 32690 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:13:04.135205 32690 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:13:04.135300 32690 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:13:04.142999 32690 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:13:04.143126 32690 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:13:04.143174 32690 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:13:04.143232 32690 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:13:04.143283 32690 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:13:04.153445 32690 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:34169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439250192320-29290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439250192320-29290-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.154.190:34389
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab
--principal=kudu/127.28.154.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.154.190:34389
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.154.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439250192320-29290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:13:04.154733 32690 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:13:04.156004 32690 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:13:04.167765 32696 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:13:04.167802 32700 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:13:04.167745 32697 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:13:04.168833 32690 server_base.cc:1034] running on GCE node
I20260811 09:13:04.169658 32690 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:13:04.171171 32690 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:13:04.172449 32690 hybrid_clock.cc:648] HybridClock initialized: now 1786439584172396 us; error 36 us; skew 500 ppm
I20260811 09:13:04.172833 32690 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:13:04 dist-test-slave-xt4k krb5kdc[32667](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439584, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:13:04.176201 32690 init.cc:377] Logged in from keytab as kudu/127.28.154.190@KRBTEST.COM (short username kudu)
I20260811 09:13:04.178018 32690 webserver.cc:460] Webserver started at http://127.28.154.190:40715/ using document root <none> and password file <none>
I20260811 09:13:04.178732 32690 fs_manager.cc:359] Metadata directory not provided
I20260811 09:13:04.178857 32690 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:13:04.179167 32690 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:13:04.182320 32690 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439250192320-29290-0/minicluster-data/master-0/data/instance:
uuid: "c0b4fd5c49094daeab977848b0dd226b"
format_stamp: "Formatted at 2026-08-11 09:13:04 on dist-test-slave-xt4k"
server_key: "b40d5522d4485dc85ee0d7aa8ccf3c94"
server_key_iv: "bed4c1f511e6fa2994cae835c8711d0a"
server_key_version: "encryptionkey@0"
I20260811 09:13:04.183131 32690 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439250192320-29290-0/minicluster-data/master-0/wal/instance:
uuid: "c0b4fd5c49094daeab977848b0dd226b"
format_stamp: "Formatted at 2026-08-11 09:13:04 on dist-test-slave-xt4k"
server_key: "b40d5522d4485dc85ee0d7aa8ccf3c94"
server_key_iv: "bed4c1f511e6fa2994cae835c8711d0a"
server_key_version: "encryptionkey@0"
I20260811 09:13:04.188892 32690 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 09:13:04.192807 32714 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:13:04.194267 32690 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 09:13:04.194453 32690 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439250192320-29290-0/minicluster-data/master-0/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439250192320-29290-0/minicluster-data/master-0/wal
uuid: "c0b4fd5c49094daeab977848b0dd226b"
format_stamp: "Formatted at 2026-08-11 09:13:04 on dist-test-slave-xt4k"
server_key: "b40d5522d4485dc85ee0d7aa8ccf3c94"
server_key_iv: "bed4c1f511e6fa2994cae835c8711d0a"
server_key_version: "encryptionkey@0"
I20260811 09:13:04.194661 32690 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439250192320-29290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439250192320-29290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439250192320-29290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:13:04.219590 32690 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:13:04.224409 32690 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:13:04.224713 32690 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:13:04.242668 32690 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.190:34389
I20260811 09:13:04.242672 343 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.190:34389 every 8 connection(s)
I20260811 09:13:04.244148 32690 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
I20260811 09:13:04.244591 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 32690
I20260811 09:13:04.244773 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439250192320-29290-0/minicluster-data/master-0/wal/instance
I20260811 09:13:04.245348 29290 external_mini_cluster.cc:1428] Setting key 9e277f08fe6277e274cafd80a6e516be
I20260811 09:13:04.248795 344 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 09:13:04 dist-test-slave-xt4k krb5kdc[32667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439583, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.154.190@KRBTEST.COM
I20260811 09:13:04.259590 344 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c0b4fd5c49094daeab977848b0dd226b: Bootstrap starting.
I20260811 09:13:04.263104 344 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c0b4fd5c49094daeab977848b0dd226b: Neither blocks nor log segments found. Creating new log.
I20260811 09:13:04.264380 344 log.cc:824] T 00000000000000000000000000000000 P c0b4fd5c49094daeab977848b0dd226b: Log is configured to *not* fsync() on all Append() calls
I20260811 09:13:04.267375 347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:04.247319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44524 (local address 127.28.154.190:34389)
0811 09:13:04.248094 (+ 775us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:04.248106 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:04.248147 (+ 41us) server_negotiation.cc:407] Connection header received
0811 09:13:04.249915 (+ 1768us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:04.249937 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:04.250307 (+ 370us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:04.250653 (+ 346us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:04.252183 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:04.253592 (+ 1409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:04.254912 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:04.255569 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:04.255738 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:04.260058 (+ 4320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:13:04.260066 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:13:04.260089 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:13:04.260126 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:13:04.263146 (+ 3020us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:13:04.264000 (+ 854us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:13:04.264007 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:13:04.264013 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:13:04.264212 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:13:04.264669 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:13:04.264674 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:13:04.264677 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:13:04.265413 (+ 736us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:13:04.265721 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:04.266025 (+ 304us) server_negotiation.cc:299] Negotiation successful
0811 09:13:04.266386 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":556,"thread_start_us":326,"threads_started":1}
I20260811 09:13:04.267594 344 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c0b4fd5c49094daeab977848b0dd226b: No bootstrap required, opened a new log
I20260811 09:13:04.275729 344 raft_consensus.cc:348] T 00000000000000000000000000000000 P c0b4fd5c49094daeab977848b0dd226b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0b4fd5c49094daeab977848b0dd226b" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 34389 } }
I20260811 09:13:04.276130 344 raft_consensus.cc:374] T 00000000000000000000000000000000 P c0b4fd5c49094daeab977848b0dd226b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:13:04.276230 344 raft_consensus.cc:729] T 00000000000000000000000000000000 P c0b4fd5c49094daeab977848b0dd226b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0b4fd5c49094daeab977848b0dd226b, State: Initialized, Role: FOLLOWER
I20260811 09:13:04.276800 344 consensus_queue.cc:260] T 00000000000000000000000000000000 P c0b4fd5c49094daeab977848b0dd226b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0b4fd5c49094daeab977848b0dd226b" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 34389 } }
I20260811 09:13:04.277019 344 raft_consensus.cc:388] T 00000000000000000000000000000000 P c0b4fd5c49094daeab977848b0dd226b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:13:04.277151 344 raft_consensus.cc:482] T 00000000000000000000000000000000 P c0b4fd5c49094daeab977848b0dd226b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:13:04.277302 344 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c0b4fd5c49094daeab977848b0dd226b [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:13:04.279487 344 raft_consensus.cc:504] T 00000000000000000000000000000000 P c0b4fd5c49094daeab977848b0dd226b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0b4fd5c49094daeab977848b0dd226b" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 34389 } }
I20260811 09:13:04.279999 344 leader_election.cc:302] T 00000000000000000000000000000000 P c0b4fd5c49094daeab977848b0dd226b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c0b4fd5c49094daeab977848b0dd226b; no voters:
I20260811 09:13:04.280658 344 leader_election.cc:288] T 00000000000000000000000000000000 P c0b4fd5c49094daeab977848b0dd226b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:13:04.280781 349 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c0b4fd5c49094daeab977848b0dd226b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:13:04.281936 349 raft_consensus.cc:686] T 00000000000000000000000000000000 P c0b4fd5c49094daeab977848b0dd226b [term 1 LEADER]: Becoming Leader. State: Replica: c0b4fd5c49094daeab977848b0dd226b, State: Running, Role: LEADER
I20260811 09:13:04.282618 349 consensus_queue.cc:237] T 00000000000000000000000000000000 P c0b4fd5c49094daeab977848b0dd226b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0b4fd5c49094daeab977848b0dd226b" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 34389 } }
I20260811 09:13:04.283332 344 sys_catalog.cc:564] T 00000000000000000000000000000000 P c0b4fd5c49094daeab977848b0dd226b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:13:04.286551 352 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0b4fd5c49094daeab977848b0dd226b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c0b4fd5c49094daeab977848b0dd226b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0b4fd5c49094daeab977848b0dd226b" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 34389 } } }
I20260811 09:13:04.286149 355 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0b4fd5c49094daeab977848b0dd226b [sys.catalog]: SysCatalogTable state changed. Reason: New leader c0b4fd5c49094daeab977848b0dd226b. Latest consensus state: current_term: 1 leader_uuid: "c0b4fd5c49094daeab977848b0dd226b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0b4fd5c49094daeab977848b0dd226b" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 34389 } } }
I20260811 09:13:04.287204 352 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0b4fd5c49094daeab977848b0dd226b [sys.catalog]: This master's current role is: LEADER
I20260811 09:13:04.287235 355 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0b4fd5c49094daeab977848b0dd226b [sys.catalog]: This master's current role is: LEADER
I20260811 09:13:04.288913 365 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:13:04.294168 365 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:13:04.307302 365 catalog_manager.cc:1347] Generated new cluster ID: b264ce0d9a0f4eb399d252b03fa6ab35
I20260811 09:13:04.307436 365 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:13:04.325889 365 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:13:04.327416 365 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:13:04.341141 365 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c0b4fd5c49094daeab977848b0dd226b: Generated new TSK 0
I20260811 09:13:04.342074 365 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.28.154.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.154.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.154.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.129.keytab.
Entry for principal kudu/127.28.154.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.129.keytab.
I20260811 09:13:04.435307 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439250192320-29290-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439250192320-29290-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439250192320-29290-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439250192320-29290-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.154.129:0
--local_ip_for_outbound_sockets=127.28.154.129
--webserver_interface=127.28.154.129
--webserver_port=0
--tserver_master_addrs=127.28.154.190:34389
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.129.keytab
--principal=kudu/127.28.154.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:34169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:13:04.797573 403 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:13:04.797852 403 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:13:04.797976 403 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:13:04.809242 403 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:13:04.809398 403 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:13:04.809580 403 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.154.129
I20260811 09:13:04.819669 403 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:34169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439250192320-29290-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439250192320-29290-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.129.keytab
--principal=kudu/127.28.154.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.154.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439250192320-29290-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.154.129
--webserver_port=0
--tserver_master_addrs=127.28.154.190:34389
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.154.129
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439250192320-29290-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:13:04.821080 403 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:13:04.822638 403 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:13:04.836798 409 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:13:04.836777 417 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:13:04.836833 411 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:13:04.837854 403 server_base.cc:1034] running on GCE node
I20260811 09:13:04.838500 403 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:13:04.839443 403 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:13:04.840643 403 hybrid_clock.cc:648] HybridClock initialized: now 1786439584840583 us; error 48 us; skew 500 ppm
I20260811 09:13:04.840957 403 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:13:04 dist-test-slave-xt4k krb5kdc[32667](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439584, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:13:04.844138 403 init.cc:377] Logged in from keytab as kudu/127.28.154.129@KRBTEST.COM (short username kudu)
I20260811 09:13:04.845667 403 webserver.cc:460] Webserver started at http://127.28.154.129:43785/ using document root <none> and password file <none>
I20260811 09:13:04.846361 403 fs_manager.cc:359] Metadata directory not provided
I20260811 09:13:04.846479 403 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:13:04.846781 403 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:13:04.849726 403 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439250192320-29290-0/minicluster-data/ts-0/data/instance:
uuid: "ca3152280a5543ae9f4fa701daa2ae6a"
format_stamp: "Formatted at 2026-08-11 09:13:04 on dist-test-slave-xt4k"
server_key: "2ce0972d3a4293796adceddad9f1990d"
server_key_iv: "1d8057786a9ca7a3500d6af98c7d508f"
server_key_version: "encryptionkey@0"
I20260811 09:13:04.850482 403 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439250192320-29290-0/minicluster-data/ts-0/wal/instance:
uuid: "ca3152280a5543ae9f4fa701daa2ae6a"
format_stamp: "Formatted at 2026-08-11 09:13:04 on dist-test-slave-xt4k"
server_key: "2ce0972d3a4293796adceddad9f1990d"
server_key_iv: "1d8057786a9ca7a3500d6af98c7d508f"
server_key_version: "encryptionkey@0"
I20260811 09:13:04.855816 403 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 09:13:04.859436 430 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:13:04.860809 403 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 09:13:04.860987 403 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439250192320-29290-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439250192320-29290-0/minicluster-data/ts-0/wal
uuid: "ca3152280a5543ae9f4fa701daa2ae6a"
format_stamp: "Formatted at 2026-08-11 09:13:04 on dist-test-slave-xt4k"
server_key: "2ce0972d3a4293796adceddad9f1990d"
server_key_iv: "1d8057786a9ca7a3500d6af98c7d508f"
server_key_version: "encryptionkey@0"
I20260811 09:13:04.861174 403 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439250192320-29290-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439250192320-29290-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439250192320-29290-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:13:04.897831 403 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:13:04.902230 403 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:13:04.902523 403 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:13:04.903507 403 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:13:04.904923 403 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:13:04.905004 403 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:13:04.905136 403 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:13:04.905190 403 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:13:04.936085 403 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.129:39527
I20260811 09:13:04.936148 647 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.129:39527 every 8 connection(s)
I20260811 09:13:04.937405 403 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439250192320-29290-0/minicluster-data/ts-0/data/info.pb
I20260811 09:13:04.944137 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 403
I20260811 09:13:04.944331 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439250192320-29290-0/minicluster-data/ts-0/wal/instance
I20260811 09:13:04.944931 29290 external_mini_cluster.cc:1428] Setting key 06cabd071068b95340f6c7f0f3dbb327
Aug 11 09:13:04 dist-test-slave-xt4k krb5kdc[32667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439584, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.129@KRBTEST.COM for kudu/127.28.154.190@KRBTEST.COM
I20260811 09:13:04.958503 650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:04.939870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.129:54493 (local address 127.28.154.190:34389)
0811 09:13:04.940315 (+ 445us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:04.940320 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:04.941155 (+ 835us) server_negotiation.cc:407] Connection header received
0811 09:13:04.943042 (+ 1887us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:04.943048 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:04.943131 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:04.943229 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:04.945194 (+ 1965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:04.945967 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:04.947997 (+ 2030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:04.948454 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:04.948558 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:04.952558 (+ 4000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:13:04.952584 (+ 26us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:13:04.952588 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:13:04.952622 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:13:04.955193 (+ 2571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:13:04.956058 (+ 865us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:13:04.956063 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:13:04.956066 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:13:04.956193 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:13:04.956670 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:13:04.956674 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:13:04.956677 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:13:04.956978 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:13:04.957103 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:04.957954 (+ 851us) server_negotiation.cc:299] Negotiation successful
0811 09:13:04.958182 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":240,"threads_started":1}
I20260811 09:13:04.959760 651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:04.940306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:34389 (local address 127.28.154.129:54493)
0811 09:13:04.940997 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0811 09:13:04.941044 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 09:13:04.942768 (+ 1724us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:13:04.943470 (+ 702us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:13:04.943479 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:13:04.944143 (+ 664us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:13:04.944979 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:13:04.945007 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:13:04.946139 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:13:04.946143 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:13:04.947840 (+ 1697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:13:04.947855 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:13:04.948689 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:13:04.948695 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:13:04.948980 (+ 285us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:04.949679 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:13:04.949711 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:13:04.952283 (+ 2572us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:13:04.955350 (+ 3067us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:13:04.955360 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:13:04.955380 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:13:04.955949 (+ 569us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:13:04.956285 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:13:04.956289 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:13:04.956292 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:13:04.956571 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:13:04.957098 (+ 527us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:13:04.957106 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:13:04.957567 (+ 461us) client_negotiation.cc:769] Sending connection context
0811 09:13:04.957863 (+ 296us) client_negotiation.cc:240] Negotiation successful
0811 09:13:04.958413 (+ 550us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":256,"threads_started":1}
I20260811 09:13:04.961560 648 heartbeater.cc:347] Connected to a master server at 127.28.154.190:34389
I20260811 09:13:04.961871 648 heartbeater.cc:464] Registering TS with master...
I20260811 09:13:04.962635 648 heartbeater.cc:511] Master 127.28.154.190:34389 requested a full tablet report, sending...
I20260811 09:13:04.965121 32746 ts_manager.cc:194] Registered new tserver with Master: ca3152280a5543ae9f4fa701daa2ae6a (127.28.154.129:39527)
I20260811 09:13:04.966715 32746 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.154.129@KRBTEST.COM'} at 127.28.154.129:54493
WARNING: no policy specified for kudu/127.28.154.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.154.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.154.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.130.keytab.
Entry for principal kudu/127.28.154.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.130.keytab.
I20260811 09:13:05.016018 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439250192320-29290-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439250192320-29290-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439250192320-29290-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439250192320-29290-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.154.130:0
--local_ip_for_outbound_sockets=127.28.154.130
--webserver_interface=127.28.154.130
--webserver_port=0
--tserver_master_addrs=127.28.154.190:34389
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.130.keytab
--principal=kudu/127.28.154.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:34169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:13:05.365525 661 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:13:05.365868 661 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:13:05.366017 661 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:13:05.373560 661 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:13:05.373695 661 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:13:05.373847 661 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.154.130
I20260811 09:13:05.383939 661 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:34169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439250192320-29290-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439250192320-29290-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.130.keytab
--principal=kudu/127.28.154.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.154.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439250192320-29290-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.154.130
--webserver_port=0
--tserver_master_addrs=127.28.154.190:34389
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.154.130
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439250192320-29290-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:13:05.385311 661 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:13:05.386664 661 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:13:05.399749 676 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:13:05.400012 671 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:13:05.399968 673 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:13:05.400681 661 server_base.cc:1034] running on GCE node
I20260811 09:13:05.401319 661 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:13:05.402302 661 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:13:05.403518 661 hybrid_clock.cc:648] HybridClock initialized: now 1786439585403440 us; error 70 us; skew 500 ppm
I20260811 09:13:05.403841 661 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:13:05 dist-test-slave-xt4k krb5kdc[32667](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439585, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:13:05.407027 661 init.cc:377] Logged in from keytab as kudu/127.28.154.130@KRBTEST.COM (short username kudu)
I20260811 09:13:05.408597 661 webserver.cc:460] Webserver started at http://127.28.154.130:46299/ using document root <none> and password file <none>
I20260811 09:13:05.409294 661 fs_manager.cc:359] Metadata directory not provided
I20260811 09:13:05.409396 661 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:13:05.409708 661 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:13:05.412801 661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439250192320-29290-0/minicluster-data/ts-1/data/instance:
uuid: "9b37be36c5c045a3a28212276e28b069"
format_stamp: "Formatted at 2026-08-11 09:13:05 on dist-test-slave-xt4k"
server_key: "4b3675cd9ffdb0eb0f60b710b5316326"
server_key_iv: "290550bca17ee8526ec9164052613f48"
server_key_version: "encryptionkey@0"
I20260811 09:13:05.413632 661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439250192320-29290-0/minicluster-data/ts-1/wal/instance:
uuid: "9b37be36c5c045a3a28212276e28b069"
format_stamp: "Formatted at 2026-08-11 09:13:05 on dist-test-slave-xt4k"
server_key: "4b3675cd9ffdb0eb0f60b710b5316326"
server_key_iv: "290550bca17ee8526ec9164052613f48"
server_key_version: "encryptionkey@0"
I20260811 09:13:05.418883 661 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 09:13:05.422421 687 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:13:05.423892 661 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 09:13:05.424041 661 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439250192320-29290-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439250192320-29290-0/minicluster-data/ts-1/wal
uuid: "9b37be36c5c045a3a28212276e28b069"
format_stamp: "Formatted at 2026-08-11 09:13:05 on dist-test-slave-xt4k"
server_key: "4b3675cd9ffdb0eb0f60b710b5316326"
server_key_iv: "290550bca17ee8526ec9164052613f48"
server_key_version: "encryptionkey@0"
I20260811 09:13:05.424226 661 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439250192320-29290-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439250192320-29290-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439250192320-29290-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:13:05.442121 661 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:13:05.446779 661 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:13:05.447063 661 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:13:05.448040 661 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:13:05.449443 661 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:13:05.449522 661 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:13:05.449656 661 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:13:05.449710 661 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:13:05.478796 661 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.130:37099
I20260811 09:13:05.478833 903 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.130:37099 every 8 connection(s)
I20260811 09:13:05.480111 661 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439250192320-29290-0/minicluster-data/ts-1/data/info.pb
I20260811 09:13:05.488741 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 661
I20260811 09:13:05.488910 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439250192320-29290-0/minicluster-data/ts-1/wal/instance
I20260811 09:13:05.489518 29290 external_mini_cluster.cc:1428] Setting key 611c5fe7b5d79ac1254a9d3a9f1b490c
Aug 11 09:13:05 dist-test-slave-xt4k krb5kdc[32667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439585, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.130@KRBTEST.COM for kudu/127.28.154.190@KRBTEST.COM
I20260811 09:13:05.502167 913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:05.482737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.130:55985 (local address 127.28.154.190:34389)
0811 09:13:05.483166 (+ 429us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:05.483170 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:05.484245 (+ 1075us) server_negotiation.cc:407] Connection header received
0811 09:13:05.486198 (+ 1953us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:05.486203 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:05.486276 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:05.486375 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:05.488048 (+ 1673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:05.488920 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:05.490988 (+ 2068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:05.491616 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:05.491730 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:05.496025 (+ 4295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:13:05.496030 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:13:05.496034 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:13:05.496084 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:13:05.498825 (+ 2741us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:13:05.499752 (+ 927us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:13:05.499757 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:13:05.499759 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:13:05.499904 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:13:05.500389 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:13:05.500393 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:13:05.500395 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:13:05.500695 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:13:05.500817 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:05.501695 (+ 878us) server_negotiation.cc:299] Negotiation successful
0811 09:13:05.501881 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":207,"threads_started":1}
I20260811 09:13:05.503497 914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:05.483316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:34389 (local address 127.28.154.130:55985)
0811 09:13:05.484080 (+ 764us) negotiation.cc:107] Waiting for socket to connect
0811 09:13:05.484125 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 09:13:05.485912 (+ 1787us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:13:05.486587 (+ 675us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:13:05.486596 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:13:05.487158 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:13:05.487850 (+ 692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:13:05.487876 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:13:05.489112 (+ 1236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:13:05.489118 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:13:05.490855 (+ 1737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:13:05.490868 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:13:05.491868 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:13:05.491877 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:13:05.492185 (+ 308us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:05.492952 (+ 767us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:13:05.493010 (+ 58us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:13:05.495757 (+ 2747us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:13:05.498975 (+ 3218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:13:05.498985 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:13:05.499003 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:13:05.499614 (+ 611us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:13:05.500021 (+ 407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:13:05.500025 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:13:05.500028 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:13:05.500267 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:13:05.500819 (+ 552us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:13:05.500826 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:13:05.501278 (+ 452us) client_negotiation.cc:769] Sending connection context
0811 09:13:05.501605 (+ 327us) client_negotiation.cc:240] Negotiation successful
0811 09:13:05.502137 (+ 532us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":511,"thread_start_us":309,"threads_started":1}
I20260811 09:13:05.505002 908 heartbeater.cc:347] Connected to a master server at 127.28.154.190:34389
I20260811 09:13:05.505342 908 heartbeater.cc:464] Registering TS with master...
I20260811 09:13:05.506266 908 heartbeater.cc:511] Master 127.28.154.190:34389 requested a full tablet report, sending...
I20260811 09:13:05.507932 32746 ts_manager.cc:194] Registered new tserver with Master: 9b37be36c5c045a3a28212276e28b069 (127.28.154.130:37099)
I20260811 09:13:05.508872 32746 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.154.130@KRBTEST.COM'} at 127.28.154.130:55985
WARNING: no policy specified for kudu/127.28.154.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.154.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.154.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.131.keytab.
Entry for principal kudu/127.28.154.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.131.keytab.
I20260811 09:13:05.558876 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439250192320-29290-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439250192320-29290-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439250192320-29290-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439250192320-29290-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.154.131:0
--local_ip_for_outbound_sockets=127.28.154.131
--webserver_interface=127.28.154.131
--webserver_port=0
--tserver_master_addrs=127.28.154.190:34389
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.131.keytab
--principal=kudu/127.28.154.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:34169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:13:05.901598 921 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:13:05.901893 921 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:13:05.901998 921 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:13:05.909439 921 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:13:05.909559 921 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:13:05.909673 921 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.154.131
I20260811 09:13:05.919585 921 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:34169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439250192320-29290-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439250192320-29290-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.131.keytab
--principal=kudu/127.28.154.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.154.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439250192320-29290-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.154.131
--webserver_port=0
--tserver_master_addrs=127.28.154.190:34389
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.154.131
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439250192320-29290-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:13:05.920894 921 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:13:05.922174 921 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:13:05.935088 929 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:13:05.935137 930 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:13:05.935844 921 server_base.cc:1034] running on GCE node
W20260811 09:13:05.935114 932 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:13:05.939845 921 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:13:05.941071 921 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:13:05.942303 921 hybrid_clock.cc:648] HybridClock initialized: now 1786439585942254 us; error 42 us; skew 500 ppm
I20260811 09:13:05.942628 921 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:13:05 dist-test-slave-xt4k krb5kdc[32667](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439585, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:13:05.945870 921 init.cc:377] Logged in from keytab as kudu/127.28.154.131@KRBTEST.COM (short username kudu)
I20260811 09:13:05.947448 921 webserver.cc:460] Webserver started at http://127.28.154.131:34307/ using document root <none> and password file <none>
I20260811 09:13:05.948167 921 fs_manager.cc:359] Metadata directory not provided
I20260811 09:13:05.948292 921 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:13:05.948647 921 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:13:05.951690 921 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439250192320-29290-0/minicluster-data/ts-2/data/instance:
uuid: "9b715657370a4b538249a50952aa1b0a"
format_stamp: "Formatted at 2026-08-11 09:13:05 on dist-test-slave-xt4k"
server_key: "08f5addde104806deff584f35843498a"
server_key_iv: "b10561f8b4ce00c48fe7495fb895c047"
server_key_version: "encryptionkey@0"
I20260811 09:13:05.952509 921 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439250192320-29290-0/minicluster-data/ts-2/wal/instance:
uuid: "9b715657370a4b538249a50952aa1b0a"
format_stamp: "Formatted at 2026-08-11 09:13:05 on dist-test-slave-xt4k"
server_key: "08f5addde104806deff584f35843498a"
server_key_iv: "b10561f8b4ce00c48fe7495fb895c047"
server_key_version: "encryptionkey@0"
I20260811 09:13:05.960719 921 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.007s sys 0.000s
I20260811 09:13:05.964781 952 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:13:05.966383 921 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 09:13:05.966562 921 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439250192320-29290-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439250192320-29290-0/minicluster-data/ts-2/wal
uuid: "9b715657370a4b538249a50952aa1b0a"
format_stamp: "Formatted at 2026-08-11 09:13:05 on dist-test-slave-xt4k"
server_key: "08f5addde104806deff584f35843498a"
server_key_iv: "b10561f8b4ce00c48fe7495fb895c047"
server_key_version: "encryptionkey@0"
I20260811 09:13:05.966775 921 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439250192320-29290-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439250192320-29290-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439250192320-29290-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:13:05.970335 648 heartbeater.cc:503] Master 127.28.154.190:34389 was elected leader, sending a full tablet report...
I20260811 09:13:06.007083 921 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:13:06.011708 921 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:13:06.011984 921 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:13:06.013036 921 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:13:06.014460 921 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:13:06.014550 921 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:13:06.014637 921 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:13:06.014679 921 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:13:06.048213 921 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.131:37769
I20260811 09:13:06.048276 1172 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.131:37769 every 8 connection(s)
I20260811 09:13:06.049533 921 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439250192320-29290-0/minicluster-data/ts-2/data/info.pb
I20260811 09:13:06.051316 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 921
I20260811 09:13:06.051487 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439250192320-29290-0/minicluster-data/ts-2/wal/instance
I20260811 09:13:06.051939 29290 external_mini_cluster.cc:1428] Setting key 22df87f7cb2eaa47c5dfaed9726963a0
Aug 11 09:13:06 dist-test-slave-xt4k krb5kdc[32667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439585, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.131@KRBTEST.COM for kudu/127.28.154.190@KRBTEST.COM
I20260811 09:13:06.070504 1175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:06.052144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.131:60111 (local address 127.28.154.190:34389)
0811 09:13:06.052655 (+ 511us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:06.052662 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:06.053796 (+ 1134us) server_negotiation.cc:407] Connection header received
0811 09:13:06.055701 (+ 1905us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:06.055705 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:06.055788 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:06.055880 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:06.057463 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:06.058258 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:06.059896 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:06.060313 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:06.060502 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:06.063898 (+ 3396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:13:06.063904 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:13:06.063908 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:13:06.063946 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:13:06.067185 (+ 3239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:13:06.067918 (+ 733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:13:06.067925 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:13:06.067927 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:13:06.068058 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:13:06.068515 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:13:06.068518 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:13:06.068521 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:13:06.068937 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:13:06.069103 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:06.069869 (+ 766us) server_negotiation.cc:299] Negotiation successful
0811 09:13:06.070102 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":368,"thread_start_us":256,"threads_started":1}
I20260811 09:13:06.071576 1176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:06.052791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:34389 (local address 127.28.154.131:60111)
0811 09:13:06.053635 (+ 844us) negotiation.cc:107] Waiting for socket to connect
0811 09:13:06.053681 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 09:13:06.055418 (+ 1737us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:13:06.056108 (+ 690us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:13:06.056117 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:13:06.056691 (+ 574us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:13:06.057273 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:13:06.057291 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:13:06.058437 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:13:06.058441 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:13:06.059726 (+ 1285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:13:06.059740 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:13:06.060601 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:13:06.060607 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:13:06.060843 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:06.061477 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:13:06.061500 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:13:06.063684 (+ 2184us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:13:06.067326 (+ 3642us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:13:06.067336 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:13:06.067350 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:13:06.067798 (+ 448us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:13:06.068166 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:13:06.068171 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:13:06.068173 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:13:06.068396 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:13:06.069095 (+ 699us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:13:06.069105 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:13:06.069461 (+ 356us) client_negotiation.cc:769] Sending connection context
0811 09:13:06.069740 (+ 279us) client_negotiation.cc:240] Negotiation successful
0811 09:13:06.070324 (+ 584us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":530,"thread_start_us":331,"threads_started":1}
I20260811 09:13:06.072986 1173 heartbeater.cc:347] Connected to a master server at 127.28.154.190:34389
I20260811 09:13:06.073277 1173 heartbeater.cc:464] Registering TS with master...
I20260811 09:13:06.074045 1173 heartbeater.cc:511] Master 127.28.154.190:34389 requested a full tablet report, sending...
I20260811 09:13:06.075515 32746 ts_manager.cc:194] Registered new tserver with Master: 9b715657370a4b538249a50952aa1b0a (127.28.154.131:37769)
I20260811 09:13:06.076411 32746 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.154.131@KRBTEST.COM'} at 127.28.154.131:60111
I20260811 09:13:06.079627 29290 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 09:13:06.097617 1175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:06.083984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37194 (local address 127.28.154.190:34389)
0811 09:13:06.084201 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:06.084209 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:06.084591 (+ 382us) server_negotiation.cc:407] Connection header received
0811 09:13:06.084849 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:06.084856 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:06.084929 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:06.085018 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:06.086445 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:06.087184 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:06.088909 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:06.089355 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:06.089486 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:06.092056 (+ 2570us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:13:06.092062 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:13:06.092065 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:13:06.092103 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:13:06.094805 (+ 2702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:13:06.095551 (+ 746us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:13:06.095556 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:13:06.095559 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:13:06.095693 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:13:06.096065 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:13:06.096069 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:13:06.096071 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:13:06.096470 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:13:06.096714 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:06.097068 (+ 354us) server_negotiation.cc:299] Negotiation successful
0811 09:13:06.097301 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 09:13:06.101694 32746 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37194:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 09:13:06.108860 32746 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 09:13:06.146164 1205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:06.121977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40342 (local address 127.28.154.129:39527)
0811 09:13:06.137957 (+ 15980us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:06.137966 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:06.137995 (+ 29us) server_negotiation.cc:407] Connection header received
0811 09:13:06.138107 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:06.138116 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:06.138433 (+ 317us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:06.138656 (+ 223us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:13:06.139527 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:06.141213 (+ 1686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:06.143414 (+ 2201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:06.144938 (+ 1524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:06.145177 (+ 239us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:06.145370 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:06.145613 (+ 243us) server_negotiation.cc:299] Negotiation successful
0811 09:13:06.145790 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15797,"thread_start_us":15614,"threads_started":1}
I20260811 09:13:06.146158 1209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:06.121851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.129:39527 (local address 127.0.0.1:40342)
0811 09:13:06.133088 (+ 11237us) negotiation.cc:107] Waiting for socket to connect
0811 09:13:06.133155 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0811 09:13:06.133386 (+ 231us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:13:06.138720 (+ 5334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:13:06.138728 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:13:06.138759 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:13:06.139239 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:13:06.139256 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:13:06.141391 (+ 2135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:13:06.141397 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:13:06.143242 (+ 1845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:13:06.143256 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:13:06.145151 (+ 1895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:13:06.145157 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:13:06.145358 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:06.145410 (+ 52us) client_negotiation.cc:769] Sending connection context
0811 09:13:06.145544 (+ 134us) client_negotiation.cc:240] Negotiation successful
0811 09:13:06.145764 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10921,"thread_start_us":17177,"threads_started":1}
I20260811 09:13:06.146939 1211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:06.122575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45830 (local address 127.28.154.131:37769)
0811 09:13:06.131926 (+ 9351us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:06.131937 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:06.137507 (+ 5570us) server_negotiation.cc:407] Connection header received
0811 09:13:06.138294 (+ 787us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:06.138304 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:06.138533 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:06.138728 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:13:06.139927 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:06.141546 (+ 1619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:06.143933 (+ 2387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:06.145309 (+ 1376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:06.145462 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:06.145612 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:06.146445 (+ 833us) server_negotiation.cc:299] Negotiation successful
0811 09:13:06.146587 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9142,"thread_start_us":8972,"threads_started":1}
I20260811 09:13:06.147558 1197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:06.122327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.130:37099 (local address 127.0.0.1:43726)
0811 09:13:06.139493 (+ 17166us) negotiation.cc:107] Waiting for socket to connect
0811 09:13:06.139511 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 09:13:06.139664 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:13:06.140772 (+ 1108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:13:06.140776 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:13:06.140800 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:13:06.141181 (+ 381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:13:06.141193 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:13:06.142928 (+ 1735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:13:06.142933 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:13:06.144957 (+ 2024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:13:06.144969 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:13:06.146730 (+ 1761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:13:06.146736 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:13:06.146898 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:06.146929 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 09:13:06.147027 (+ 98us) client_negotiation.cc:240] Negotiation successful
0811 09:13:06.147162 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17039,"mutex_wait_us":27,"thread_start_us":10279,"threads_started":1}
I20260811 09:13:06.147948 1199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:06.122011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.131:37769 (local address 127.0.0.1:45830)
0811 09:13:06.137369 (+ 15358us) negotiation.cc:107] Waiting for socket to connect
0811 09:13:06.137389 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 09:13:06.138101 (+ 712us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:13:06.139031 (+ 930us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:13:06.139035 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:13:06.139081 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:13:06.139677 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:13:06.139692 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:13:06.141687 (+ 1995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:13:06.141693 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:13:06.143748 (+ 2055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:13:06.143764 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:13:06.145800 (+ 2036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:13:06.145806 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:13:06.145989 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:06.146021 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 09:13:06.146107 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 09:13:06.147321 (+ 1214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15229,"mutex_wait_us":62,"thread_start_us":15107,"threads_started":1}
I20260811 09:13:06.150281 524 tablet_service.cc:1467] Processing CreateTablet for tablet e3c3f40b697c410caf8ab355878ad227 (DEFAULT_TABLE table=test-table [id=8cd53d55b0ae451bb38ecce8a0713d80]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:13:06.150949 1044 tablet_service.cc:1467] Processing CreateTablet for tablet e3c3f40b697c410caf8ab355878ad227 (DEFAULT_TABLE table=test-table [id=8cd53d55b0ae451bb38ecce8a0713d80]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:13:06.151566 524 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3c3f40b697c410caf8ab355878ad227. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:13:06.151686 1212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:06.122643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43726 (local address 127.28.154.130:37099)
0811 09:13:06.137014 (+ 14371us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:06.137023 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:06.139636 (+ 2613us) server_negotiation.cc:407] Connection header received
0811 09:13:06.140036 (+ 400us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:06.140045 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:06.140296 (+ 251us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:06.140523 (+ 227us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:13:06.141351 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:06.142780 (+ 1429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:06.145135 (+ 2355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:06.146515 (+ 1380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:06.146685 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:06.146844 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:06.147089 (+ 245us) server_negotiation.cc:299] Negotiation successful
0811 09:13:06.147335 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14170,"thread_start_us":14362,"threads_started":1}
I20260811 09:13:06.152136 1044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3c3f40b697c410caf8ab355878ad227. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:13:06.155269 766 tablet_service.cc:1467] Processing CreateTablet for tablet e3c3f40b697c410caf8ab355878ad227 (DEFAULT_TABLE table=test-table [id=8cd53d55b0ae451bb38ecce8a0713d80]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:13:06.156658 766 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3c3f40b697c410caf8ab355878ad227. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:13:06.163175 1214 tablet_bootstrap.cc:492] T e3c3f40b697c410caf8ab355878ad227 P ca3152280a5543ae9f4fa701daa2ae6a: Bootstrap starting.
I20260811 09:13:06.166846 1215 tablet_bootstrap.cc:492] T e3c3f40b697c410caf8ab355878ad227 P 9b715657370a4b538249a50952aa1b0a: Bootstrap starting.
I20260811 09:13:06.168286 1214 tablet_bootstrap.cc:654] T e3c3f40b697c410caf8ab355878ad227 P ca3152280a5543ae9f4fa701daa2ae6a: Neither blocks nor log segments found. Creating new log.
I20260811 09:13:06.169817 1214 log.cc:824] T e3c3f40b697c410caf8ab355878ad227 P ca3152280a5543ae9f4fa701daa2ae6a: Log is configured to *not* fsync() on all Append() calls
I20260811 09:13:06.170735 1215 tablet_bootstrap.cc:654] T e3c3f40b697c410caf8ab355878ad227 P 9b715657370a4b538249a50952aa1b0a: Neither blocks nor log segments found. Creating new log.
I20260811 09:13:06.171893 1215 log.cc:824] T e3c3f40b697c410caf8ab355878ad227 P 9b715657370a4b538249a50952aa1b0a: Log is configured to *not* fsync() on all Append() calls
I20260811 09:13:06.173454 1214 tablet_bootstrap.cc:492] T e3c3f40b697c410caf8ab355878ad227 P ca3152280a5543ae9f4fa701daa2ae6a: No bootstrap required, opened a new log
I20260811 09:13:06.173990 1214 ts_tablet_manager.cc:1397] T e3c3f40b697c410caf8ab355878ad227 P ca3152280a5543ae9f4fa701daa2ae6a: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.004s
I20260811 09:13:06.180279 1217 tablet_bootstrap.cc:492] T e3c3f40b697c410caf8ab355878ad227 P 9b37be36c5c045a3a28212276e28b069: Bootstrap starting.
I20260811 09:13:06.181752 1215 tablet_bootstrap.cc:492] T e3c3f40b697c410caf8ab355878ad227 P 9b715657370a4b538249a50952aa1b0a: No bootstrap required, opened a new log
I20260811 09:13:06.182073 1215 ts_tablet_manager.cc:1397] T e3c3f40b697c410caf8ab355878ad227 P 9b715657370a4b538249a50952aa1b0a: Time spent bootstrapping tablet: real 0.015s user 0.005s sys 0.008s
I20260811 09:13:06.183881 1217 tablet_bootstrap.cc:654] T e3c3f40b697c410caf8ab355878ad227 P 9b37be36c5c045a3a28212276e28b069: Neither blocks nor log segments found. Creating new log.
I20260811 09:13:06.184998 1217 log.cc:824] T e3c3f40b697c410caf8ab355878ad227 P 9b37be36c5c045a3a28212276e28b069: Log is configured to *not* fsync() on all Append() calls
I20260811 09:13:06.185372 1214 raft_consensus.cc:348] T e3c3f40b697c410caf8ab355878ad227 P ca3152280a5543ae9f4fa701daa2ae6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca3152280a5543ae9f4fa701daa2ae6a" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 39527 } } peers { permanent_uuid: "9b715657370a4b538249a50952aa1b0a" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 37769 } } peers { permanent_uuid: "9b37be36c5c045a3a28212276e28b069" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 37099 } }
I20260811 09:13:06.185830 1214 raft_consensus.cc:374] T e3c3f40b697c410caf8ab355878ad227 P ca3152280a5543ae9f4fa701daa2ae6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:13:06.185932 1214 raft_consensus.cc:729] T e3c3f40b697c410caf8ab355878ad227 P ca3152280a5543ae9f4fa701daa2ae6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca3152280a5543ae9f4fa701daa2ae6a, State: Initialized, Role: FOLLOWER
I20260811 09:13:06.186584 1214 consensus_queue.cc:260] T e3c3f40b697c410caf8ab355878ad227 P ca3152280a5543ae9f4fa701daa2ae6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca3152280a5543ae9f4fa701daa2ae6a" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 39527 } } peers { permanent_uuid: "9b715657370a4b538249a50952aa1b0a" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 37769 } } peers { permanent_uuid: "9b37be36c5c045a3a28212276e28b069" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 37099 } }
I20260811 09:13:06.188012 1217 tablet_bootstrap.cc:492] T e3c3f40b697c410caf8ab355878ad227 P 9b37be36c5c045a3a28212276e28b069: No bootstrap required, opened a new log
I20260811 09:13:06.188302 1217 ts_tablet_manager.cc:1397] T e3c3f40b697c410caf8ab355878ad227 P 9b37be36c5c045a3a28212276e28b069: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260811 09:13:06.193032 1215 raft_consensus.cc:348] T e3c3f40b697c410caf8ab355878ad227 P 9b715657370a4b538249a50952aa1b0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca3152280a5543ae9f4fa701daa2ae6a" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 39527 } } peers { permanent_uuid: "9b715657370a4b538249a50952aa1b0a" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 37769 } } peers { permanent_uuid: "9b37be36c5c045a3a28212276e28b069" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 37099 } }
I20260811 09:13:06.193507 1215 raft_consensus.cc:374] T e3c3f40b697c410caf8ab355878ad227 P 9b715657370a4b538249a50952aa1b0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:13:06.193603 1215 raft_consensus.cc:729] T e3c3f40b697c410caf8ab355878ad227 P 9b715657370a4b538249a50952aa1b0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b715657370a4b538249a50952aa1b0a, State: Initialized, Role: FOLLOWER
I20260811 09:13:06.193946 1214 ts_tablet_manager.cc:1428] T e3c3f40b697c410caf8ab355878ad227 P ca3152280a5543ae9f4fa701daa2ae6a: Time spent starting tablet: real 0.020s user 0.012s sys 0.007s
I20260811 09:13:06.194324 1215 consensus_queue.cc:260] T e3c3f40b697c410caf8ab355878ad227 P 9b715657370a4b538249a50952aa1b0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca3152280a5543ae9f4fa701daa2ae6a" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 39527 } } peers { permanent_uuid: "9b715657370a4b538249a50952aa1b0a" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 37769 } } peers { permanent_uuid: "9b37be36c5c045a3a28212276e28b069" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 37099 } }
I20260811 09:13:06.195446 1173 heartbeater.cc:503] Master 127.28.154.190:34389 was elected leader, sending a full tablet report...
I20260811 09:13:06.196113 1215 ts_tablet_manager.cc:1428] T e3c3f40b697c410caf8ab355878ad227 P 9b715657370a4b538249a50952aa1b0a: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
I20260811 09:13:06.197289 1217 raft_consensus.cc:348] T e3c3f40b697c410caf8ab355878ad227 P 9b37be36c5c045a3a28212276e28b069 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca3152280a5543ae9f4fa701daa2ae6a" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 39527 } } peers { permanent_uuid: "9b715657370a4b538249a50952aa1b0a" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 37769 } } peers { permanent_uuid: "9b37be36c5c045a3a28212276e28b069" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 37099 } }
I20260811 09:13:06.197757 1217 raft_consensus.cc:374] T e3c3f40b697c410caf8ab355878ad227 P 9b37be36c5c045a3a28212276e28b069 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:13:06.197865 1217 raft_consensus.cc:729] T e3c3f40b697c410caf8ab355878ad227 P 9b37be36c5c045a3a28212276e28b069 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b37be36c5c045a3a28212276e28b069, State: Initialized, Role: FOLLOWER
I20260811 09:13:06.198550 1217 consensus_queue.cc:260] T e3c3f40b697c410caf8ab355878ad227 P 9b37be36c5c045a3a28212276e28b069 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca3152280a5543ae9f4fa701daa2ae6a" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 39527 } } peers { permanent_uuid: "9b715657370a4b538249a50952aa1b0a" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 37769 } } peers { permanent_uuid: "9b37be36c5c045a3a28212276e28b069" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 37099 } }
I20260811 09:13:06.199609 908 heartbeater.cc:503] Master 127.28.154.190:34389 was elected leader, sending a full tablet report...
I20260811 09:13:06.200388 1217 ts_tablet_manager.cc:1428] T e3c3f40b697c410caf8ab355878ad227 P 9b37be36c5c045a3a28212276e28b069: Time spent starting tablet: real 0.012s user 0.011s sys 0.001s
I20260811 09:13:06.201625 1227 raft_consensus.cc:482] T e3c3f40b697c410caf8ab355878ad227 P ca3152280a5543ae9f4fa701daa2ae6a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:13:06.201857 1227 raft_consensus.cc:504] T e3c3f40b697c410caf8ab355878ad227 P ca3152280a5543ae9f4fa701daa2ae6a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca3152280a5543ae9f4fa701daa2ae6a" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 39527 } } peers { permanent_uuid: "9b715657370a4b538249a50952aa1b0a" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 37769 } } peers { permanent_uuid: "9b37be36c5c045a3a28212276e28b069" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 37099 } }
I20260811 09:13:06.203657 1227 leader_election.cc:288] T e3c3f40b697c410caf8ab355878ad227 P ca3152280a5543ae9f4fa701daa2ae6a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9b715657370a4b538249a50952aa1b0a (127.28.154.131:37769), 9b37be36c5c045a3a28212276e28b069 (127.28.154.130:37099)
I20260811 09:13:06.212617 1212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:06.204998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.129:48895 (local address 127.28.154.130:37099)
0811 09:13:06.205194 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:06.205200 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:06.205216 (+ 16us) server_negotiation.cc:407] Connection header received
0811 09:13:06.205273 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:06.205277 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:06.205346 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:06.205426 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:13:06.206063 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:06.207103 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:06.210149 (+ 3046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:06.211373 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:06.211539 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:06.211637 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:06.212148 (+ 511us) server_negotiation.cc:299] Negotiation successful
0811 09:13:06.212289 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 09:13:06.212589 1211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:06.204762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.129:43089 (local address 127.28.154.131:37769)
0811 09:13:06.204968 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:06.204973 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:06.204988 (+ 15us) server_negotiation.cc:407] Connection header received
0811 09:13:06.205045 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:06.205051 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:06.205137 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:06.205232 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:13:06.205910 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:06.207087 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:06.210139 (+ 3052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:06.211376 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:06.211542 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:06.211620 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:06.212148 (+ 528us) server_negotiation.cc:299] Negotiation successful
0811 09:13:06.212289 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 09:13:06.212815 1230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:06.203898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.131:37769 (local address 127.28.154.129:43089)
0811 09:13:06.204663 (+ 765us) negotiation.cc:107] Waiting for socket to connect
0811 09:13:06.204686 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 09:13:06.204799 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:13:06.205261 (+ 462us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:13:06.205266 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:13:06.205296 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:13:06.205745 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:13:06.205758 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:13:06.207257 (+ 1499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:13:06.207263 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:13:06.209971 (+ 2708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:13:06.209981 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:13:06.211668 (+ 1687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:13:06.211677 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:13:06.211905 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:06.212005 (+ 100us) client_negotiation.cc:769] Sending connection context
0811 09:13:06.212104 (+ 99us) client_negotiation.cc:240] Negotiation successful
0811 09:13:06.212308 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":626,"mutex_wait_us":239,"thread_start_us":384,"threads_started":1}
I20260811 09:13:06.213169 1232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:06.204090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.130:37099 (local address 127.28.154.129:48895)
0811 09:13:06.204836 (+ 746us) negotiation.cc:107] Waiting for socket to connect
0811 09:13:06.204854 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 09:13:06.204986 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:13:06.205471 (+ 485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:13:06.205476 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:13:06.205496 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:13:06.205920 (+ 424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:13:06.205932 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:13:06.207259 (+ 1327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:13:06.207263 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:13:06.209954 (+ 2691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:13:06.209976 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:13:06.211650 (+ 1674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:13:06.211659 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:13:06.211904 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:06.212005 (+ 101us) client_negotiation.cc:769] Sending connection context
0811 09:13:06.212105 (+ 100us) client_negotiation.cc:240] Negotiation successful
0811 09:13:06.212309 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":625,"mutex_wait_us":235,"thread_start_us":957,"threads_started":1}
I20260811 09:13:06.213960 1080 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3c3f40b697c410caf8ab355878ad227" candidate_uuid: "ca3152280a5543ae9f4fa701daa2ae6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b715657370a4b538249a50952aa1b0a" is_pre_election: true
I20260811 09:13:06.214148 807 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3c3f40b697c410caf8ab355878ad227" candidate_uuid: "ca3152280a5543ae9f4fa701daa2ae6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b37be36c5c045a3a28212276e28b069" is_pre_election: true
I20260811 09:13:06.214512 1080 raft_consensus.cc:2454] T e3c3f40b697c410caf8ab355878ad227 P 9b715657370a4b538249a50952aa1b0a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca3152280a5543ae9f4fa701daa2ae6a in term 0.
I20260811 09:13:06.214672 807 raft_consensus.cc:2454] T e3c3f40b697c410caf8ab355878ad227 P 9b37be36c5c045a3a28212276e28b069 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ca3152280a5543ae9f4fa701daa2ae6a in term 0.
I20260811 09:13:06.215229 438 leader_election.cc:302] T e3c3f40b697c410caf8ab355878ad227 P ca3152280a5543ae9f4fa701daa2ae6a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9b715657370a4b538249a50952aa1b0a, ca3152280a5543ae9f4fa701daa2ae6a; no voters:
I20260811 09:13:06.215652 1227 raft_consensus.cc:2781] T e3c3f40b697c410caf8ab355878ad227 P ca3152280a5543ae9f4fa701daa2ae6a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 09:13:06.215826 1227 raft_consensus.cc:482] T e3c3f40b697c410caf8ab355878ad227 P ca3152280a5543ae9f4fa701daa2ae6a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 09:13:06.215937 1227 raft_consensus.cc:3037] T e3c3f40b697c410caf8ab355878ad227 P ca3152280a5543ae9f4fa701daa2ae6a [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:13:06.218420 1227 raft_consensus.cc:504] T e3c3f40b697c410caf8ab355878ad227 P ca3152280a5543ae9f4fa701daa2ae6a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca3152280a5543ae9f4fa701daa2ae6a" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 39527 } } peers { permanent_uuid: "9b715657370a4b538249a50952aa1b0a" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 37769 } } peers { permanent_uuid: "9b37be36c5c045a3a28212276e28b069" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 37099 } }
I20260811 09:13:06.219081 1227 leader_election.cc:288] T e3c3f40b697c410caf8ab355878ad227 P ca3152280a5543ae9f4fa701daa2ae6a [CANDIDATE]: Term 1 election: Requested vote from peers 9b715657370a4b538249a50952aa1b0a (127.28.154.131:37769), 9b37be36c5c045a3a28212276e28b069 (127.28.154.130:37099)
I20260811 09:13:06.219506 1080 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3c3f40b697c410caf8ab355878ad227" candidate_uuid: "ca3152280a5543ae9f4fa701daa2ae6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b715657370a4b538249a50952aa1b0a"
I20260811 09:13:06.219601 807 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3c3f40b697c410caf8ab355878ad227" candidate_uuid: "ca3152280a5543ae9f4fa701daa2ae6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9b37be36c5c045a3a28212276e28b069"
I20260811 09:13:06.219745 1080 raft_consensus.cc:3037] T e3c3f40b697c410caf8ab355878ad227 P 9b715657370a4b538249a50952aa1b0a [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:13:06.219799 807 raft_consensus.cc:3037] T e3c3f40b697c410caf8ab355878ad227 P 9b37be36c5c045a3a28212276e28b069 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:13:06.222980 807 raft_consensus.cc:2454] T e3c3f40b697c410caf8ab355878ad227 P 9b37be36c5c045a3a28212276e28b069 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca3152280a5543ae9f4fa701daa2ae6a in term 1.
I20260811 09:13:06.222983 1080 raft_consensus.cc:2454] T e3c3f40b697c410caf8ab355878ad227 P 9b715657370a4b538249a50952aa1b0a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ca3152280a5543ae9f4fa701daa2ae6a in term 1.
I20260811 09:13:06.223423 438 leader_election.cc:302] T e3c3f40b697c410caf8ab355878ad227 P ca3152280a5543ae9f4fa701daa2ae6a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 9b37be36c5c045a3a28212276e28b069, 9b715657370a4b538249a50952aa1b0a, ca3152280a5543ae9f4fa701daa2ae6a; no voters:
I20260811 09:13:06.223740 1227 raft_consensus.cc:2781] T e3c3f40b697c410caf8ab355878ad227 P ca3152280a5543ae9f4fa701daa2ae6a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:13:06.224197 1227 raft_consensus.cc:686] T e3c3f40b697c410caf8ab355878ad227 P ca3152280a5543ae9f4fa701daa2ae6a [term 1 LEADER]: Becoming Leader. State: Replica: ca3152280a5543ae9f4fa701daa2ae6a, State: Running, Role: LEADER
I20260811 09:13:06.224776 1227 consensus_queue.cc:237] T e3c3f40b697c410caf8ab355878ad227 P ca3152280a5543ae9f4fa701daa2ae6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca3152280a5543ae9f4fa701daa2ae6a" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 39527 } } peers { permanent_uuid: "9b715657370a4b538249a50952aa1b0a" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 37769 } } peers { permanent_uuid: "9b37be36c5c045a3a28212276e28b069" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 37099 } }
I20260811 09:13:06.229574 32745 catalog_manager.cc:5495] T e3c3f40b697c410caf8ab355878ad227 P ca3152280a5543ae9f4fa701daa2ae6a reported cstate change: term changed from 0 to 1, leader changed from <none> to ca3152280a5543ae9f4fa701daa2ae6a (127.28.154.129). New cstate: current_term: 1 leader_uuid: "ca3152280a5543ae9f4fa701daa2ae6a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ca3152280a5543ae9f4fa701daa2ae6a" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 39527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9b715657370a4b538249a50952aa1b0a" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 37769 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9b37be36c5c045a3a28212276e28b069" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 37099 } health_report { overall_health: UNKNOWN } } }
W20260811 09:13:06.231990 910 tablet.cc:2369] T e3c3f40b697c410caf8ab355878ad227 P 9b37be36c5c045a3a28212276e28b069: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 09:13:06.248266 1205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:06.242942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40350 (local address 127.28.154.129:39527)
0811 09:13:06.243179 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:06.243184 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:06.243204 (+ 20us) server_negotiation.cc:407] Connection header received
0811 09:13:06.243376 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:06.243381 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:06.243473 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:06.243629 (+ 156us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:13:06.243979 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:06.244755 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:06.246099 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:06.246500 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:06.246614 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:06.247119 (+ 505us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:13:06.247674 (+ 555us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:13:06.247810 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:06.247918 (+ 108us) server_negotiation.cc:299] Negotiation successful
0811 09:13:06.248006 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 09:13:06.257252 807 raft_consensus.cc:1261] T e3c3f40b697c410caf8ab355878ad227 P 9b37be36c5c045a3a28212276e28b069 [term 1 FOLLOWER]: Refusing update from remote peer ca3152280a5543ae9f4fa701daa2ae6a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:13:06.257368 1080 raft_consensus.cc:1261] T e3c3f40b697c410caf8ab355878ad227 P 9b715657370a4b538249a50952aa1b0a [term 1 FOLLOWER]: Refusing update from remote peer ca3152280a5543ae9f4fa701daa2ae6a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:13:06.258317 1227 consensus_queue.cc:1035] T e3c3f40b697c410caf8ab355878ad227 P ca3152280a5543ae9f4fa701daa2ae6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b37be36c5c045a3a28212276e28b069" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 37099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:13:06.258636 1234 consensus_queue.cc:1035] T e3c3f40b697c410caf8ab355878ad227 P ca3152280a5543ae9f4fa701daa2ae6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "9b715657370a4b538249a50952aa1b0a" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 37769 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:13:06.272630 1238 mvcc.cc:205] Tried to move back new op lower bound from 7317256545298587648 to 7317256545180196864. Current Snapshot: MvccSnapshot[applied={T|T < 7317256545298587648}]
I20260811 09:13:06.289336 32746 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37194:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:13:06.289682 32746 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37194:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:13:06.294166 32746 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c0b4fd5c49094daeab977848b0dd226b: Sending DeleteTablet for 3 replicas of tablet e3c3f40b697c410caf8ab355878ad227
I20260811 09:13:06.295507 524 tablet_service.cc:1514] Processing DeleteTablet for tablet e3c3f40b697c410caf8ab355878ad227 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:13:06 UTC) from {username='kudu', principal='kudu/127.28.154.190@KRBTEST.COM'} at 127.0.0.1:40342
I20260811 09:13:06.295985 766 tablet_service.cc:1514] Processing DeleteTablet for tablet e3c3f40b697c410caf8ab355878ad227 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:13:06 UTC) from {username='kudu', principal='kudu/127.28.154.190@KRBTEST.COM'} at 127.0.0.1:43726
I20260811 09:13:06.296106 1044 tablet_service.cc:1514] Processing DeleteTablet for tablet e3c3f40b697c410caf8ab355878ad227 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:13:06 UTC) from {username='kudu', principal='kudu/127.28.154.190@KRBTEST.COM'} at 127.0.0.1:45830
I20260811 09:13:06.303469 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 403
I20260811 09:13:06.313925 1258 tablet_replica.cc:330] T e3c3f40b697c410caf8ab355878ad227 P ca3152280a5543ae9f4fa701daa2ae6a: stopping tablet replica
I20260811 09:13:06.314188 1262 tablet_replica.cc:330] T e3c3f40b697c410caf8ab355878ad227 P 9b715657370a4b538249a50952aa1b0a: stopping tablet replica
I20260811 09:13:06.314739 1258 raft_consensus.cc:2229] T e3c3f40b697c410caf8ab355878ad227 P ca3152280a5543ae9f4fa701daa2ae6a [term 1 LEADER]: Raft consensus shutting down.
I20260811 09:13:06.314813 1262 raft_consensus.cc:2229] T e3c3f40b697c410caf8ab355878ad227 P 9b715657370a4b538249a50952aa1b0a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:13:06.315516 1262 raft_consensus.cc:2258] T e3c3f40b697c410caf8ab355878ad227 P 9b715657370a4b538249a50952aa1b0a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:13:06.316532 1260 tablet_replica.cc:330] T e3c3f40b697c410caf8ab355878ad227 P 9b37be36c5c045a3a28212276e28b069: stopping tablet replica
I20260811 09:13:06.316931 1260 raft_consensus.cc:2229] T e3c3f40b697c410caf8ab355878ad227 P 9b37be36c5c045a3a28212276e28b069 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:13:06.317414 1260 raft_consensus.cc:2258] T e3c3f40b697c410caf8ab355878ad227 P 9b37be36c5c045a3a28212276e28b069 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:13:06.317605 1258 raft_consensus.cc:2258] T e3c3f40b697c410caf8ab355878ad227 P ca3152280a5543ae9f4fa701daa2ae6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:13:06.319936 1260 ts_tablet_manager.cc:1905] T e3c3f40b697c410caf8ab355878ad227 P 9b37be36c5c045a3a28212276e28b069: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:13:06.319986 1258 ts_tablet_manager.cc:1905] T e3c3f40b697c410caf8ab355878ad227 P ca3152280a5543ae9f4fa701daa2ae6a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:13:06.320369 1262 ts_tablet_manager.cc:1905] T e3c3f40b697c410caf8ab355878ad227 P 9b715657370a4b538249a50952aa1b0a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:13:06.327683 1262 ts_tablet_manager.cc:1918] T e3c3f40b697c410caf8ab355878ad227 P 9b715657370a4b538249a50952aa1b0a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:13:06.327908 1262 log.cc:1196] T e3c3f40b697c410caf8ab355878ad227 P 9b715657370a4b538249a50952aa1b0a: Deleting WAL directory at /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439250192320-29290-0/minicluster-data/ts-2/wal/wals/e3c3f40b697c410caf8ab355878ad227
I20260811 09:13:06.328097 1258 ts_tablet_manager.cc:1918] T e3c3f40b697c410caf8ab355878ad227 P ca3152280a5543ae9f4fa701daa2ae6a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:13:06.328330 1258 log.cc:1196] T e3c3f40b697c410caf8ab355878ad227 P ca3152280a5543ae9f4fa701daa2ae6a: Deleting WAL directory at /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439250192320-29290-0/minicluster-data/ts-0/wal/wals/e3c3f40b697c410caf8ab355878ad227
I20260811 09:13:06.328387 1260 ts_tablet_manager.cc:1918] T e3c3f40b697c410caf8ab355878ad227 P 9b37be36c5c045a3a28212276e28b069: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:13:06.328732 1260 log.cc:1196] T e3c3f40b697c410caf8ab355878ad227 P 9b37be36c5c045a3a28212276e28b069: Deleting WAL directory at /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786439250192320-29290-0/minicluster-data/ts-1/wal/wals/e3c3f40b697c410caf8ab355878ad227
I20260811 09:13:06.329018 1258 ts_tablet_manager.cc:1939] T e3c3f40b697c410caf8ab355878ad227 P ca3152280a5543ae9f4fa701daa2ae6a: Deleting consensus metadata
I20260811 09:13:06.329403 1260 ts_tablet_manager.cc:1939] T e3c3f40b697c410caf8ab355878ad227 P 9b37be36c5c045a3a28212276e28b069: Deleting consensus metadata
I20260811 09:13:06.330643 1262 ts_tablet_manager.cc:1939] T e3c3f40b697c410caf8ab355878ad227 P 9b715657370a4b538249a50952aa1b0a: Deleting consensus metadata
Aug 11 09:13:06 dist-test-slave-xt4k krb5kdc[32667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439583, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.154.129@KRBTEST.COM
I20260811 09:13:06.331964 32715 catalog_manager.cc:4841] TS 9b37be36c5c045a3a28212276e28b069 (127.28.154.130:37099): tablet e3c3f40b697c410caf8ab355878ad227 (table test-table [id=8cd53d55b0ae451bb38ecce8a0713d80]) successfully deleted
I20260811 09:13:06.332387 32719 catalog_manager.cc:4841] TS 9b715657370a4b538249a50952aa1b0a (127.28.154.131:37769): tablet e3c3f40b697c410caf8ab355878ad227 (table test-table [id=8cd53d55b0ae451bb38ecce8a0713d80]) successfully deleted
I20260811 09:13:06.333330 32722 catalog_manager.cc:4841] TS ca3152280a5543ae9f4fa701daa2ae6a (127.28.154.129:39527): tablet e3c3f40b697c410caf8ab355878ad227 (table test-table [id=8cd53d55b0ae451bb38ecce8a0713d80]) successfully deleted
I20260811 09:13:06.339808 1205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:06.305363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40366 (local address 127.28.154.129:39527)
0811 09:13:06.305535 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:06.305541 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:06.305564 (+ 23us) server_negotiation.cc:407] Connection header received
0811 09:13:06.305662 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:06.305667 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:06.305752 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:06.305866 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:06.309175 (+ 3309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:06.310433 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:06.315035 (+ 4602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:06.315785 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:06.315906 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:06.333053 (+ 17147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:13:06.333068 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:13:06.333088 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:13:06.333138 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:13:06.336189 (+ 3051us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:13:06.337430 (+ 1241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:13:06.337439 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:13:06.337441 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:13:06.337628 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:13:06.338141 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:13:06.338145 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:13:06.338147 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:13:06.338563 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:13:06.338811 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:06.339313 (+ 502us) server_negotiation.cc:299] Negotiation successful
0811 09:13:06.339514 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 09:13:06.340967 641 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:13:06.442732 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 403
I20260811 09:13:06.472692 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 661
Aug 11 09:13:06 dist-test-slave-xt4k krb5kdc[32667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439583, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.154.130@KRBTEST.COM
I20260811 09:13:06.489228 1212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:06.473491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43740 (local address 127.28.154.130:37099)
0811 09:13:06.473650 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:06.473655 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:06.473673 (+ 18us) server_negotiation.cc:407] Connection header received
0811 09:13:06.473778 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:06.473782 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:06.473846 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:06.473927 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:06.475280 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:06.476091 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:06.477384 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:06.477807 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:06.477910 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:06.483390 (+ 5480us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:13:06.483403 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:13:06.483422 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:13:06.483464 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:13:06.486467 (+ 3003us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:13:06.487289 (+ 822us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:13:06.487296 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:13:06.487298 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:13:06.487419 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:13:06.487741 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:13:06.487744 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:13:06.487746 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:13:06.488170 (+ 424us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:13:06.488432 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:06.488746 (+ 314us) server_negotiation.cc:299] Negotiation successful
0811 09:13:06.488915 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 09:13:06.490150 894 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:13:06.578895 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 661
I20260811 09:13:06.604074 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 921
Aug 11 09:13:06 dist-test-slave-xt4k krb5kdc[32667](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439583, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.154.131@KRBTEST.COM
I20260811 09:13:06.618353 1211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:06.604920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45846 (local address 127.28.154.131:37769)
0811 09:13:06.605073 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:06.605078 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:06.605097 (+ 19us) server_negotiation.cc:407] Connection header received
0811 09:13:06.605215 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:06.605219 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:06.605284 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:06.605369 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:06.606653 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:06.607369 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:06.608678 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:06.609113 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:06.609212 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:06.613428 (+ 4216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:13:06.613445 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:13:06.613462 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:13:06.613501 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:13:06.616034 (+ 2533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:13:06.616710 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:13:06.616717 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:13:06.616719 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:13:06.616850 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:13:06.617133 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:13:06.617136 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:13:06.617138 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:13:06.617479 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:13:06.617740 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:06.617932 (+ 192us) server_negotiation.cc:299] Negotiation successful
0811 09:13:06.618106 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 09:13:06.619004 1162 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:13:06.708272 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 921
I20260811 09:13:06.734103 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 32690
I20260811 09:13:06.735184 331 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:13:06.860548 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 32690
2026-08-11T09:13:06Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (4697 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-11T09:13:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:13:06Z Disabled control of system clock
I20260811 09:13:06.954388 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439250192320-29290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439250192320-29290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439250192320-29290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.154.190:34443
--webserver_interface=127.28.154.190
--webserver_port=0
--builtin_ntp_servers=127.28.154.148:46103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.154.190:34443
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 09:13:07.297402 1276 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:13:07.297727 1276 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:13:07.297860 1276 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:13:07.305222 1276 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:13:07.305352 1276 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:13:07.305457 1276 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:13:07.305547 1276 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:13:07.305606 1276 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:13:07.315025 1276 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:46103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439250192320-29290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439250192320-29290-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.154.190:34443
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.154.190:34443
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.154.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439250192320-29290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:13:07.316653 1276 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:13:07.318483 1276 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:13:07.330039 1293 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:13:07.330096 1286 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:13:07.330122 1289 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:13:07.330984 1276 server_base.cc:1034] running on GCE node
I20260811 09:13:07.331789 1276 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:13:07.333292 1276 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:13:07.334533 1276 hybrid_clock.cc:648] HybridClock initialized: now 1786439587334473 us; error 49 us; skew 500 ppm
I20260811 09:13:07.334878 1276 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 09:13:07.341184 1276 webserver.cc:460] Webserver started at http://127.28.154.190:39911/ using document root <none> and password file <none>
I20260811 09:13:07.341926 1276 fs_manager.cc:359] Metadata directory not provided
I20260811 09:13:07.342022 1276 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:13:07.342309 1276 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:13:07.345342 1276 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439250192320-29290-0/minicluster-data/master-0/data/instance:
uuid: "e8cde04e09534d9887748b52b4357b16"
format_stamp: "Formatted at 2026-08-11 09:13:07 on dist-test-slave-xt4k"
server_key: "ac335a6baf2cff284b503b9bf6a6022b"
server_key_iv: "92a7f8830beb8098e2cdd55b6876641c"
server_key_version: "encryptionkey@0"
I20260811 09:13:07.346109 1276 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439250192320-29290-0/minicluster-data/master-0/wal/instance:
uuid: "e8cde04e09534d9887748b52b4357b16"
format_stamp: "Formatted at 2026-08-11 09:13:07 on dist-test-slave-xt4k"
server_key: "ac335a6baf2cff284b503b9bf6a6022b"
server_key_iv: "92a7f8830beb8098e2cdd55b6876641c"
server_key_version: "encryptionkey@0"
I20260811 09:13:07.351346 1276 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260811 09:13:07.354943 1307 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:13:07.356323 1276 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 09:13:07.356581 1276 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439250192320-29290-0/minicluster-data/master-0/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439250192320-29290-0/minicluster-data/master-0/wal
uuid: "e8cde04e09534d9887748b52b4357b16"
format_stamp: "Formatted at 2026-08-11 09:13:07 on dist-test-slave-xt4k"
server_key: "ac335a6baf2cff284b503b9bf6a6022b"
server_key_iv: "92a7f8830beb8098e2cdd55b6876641c"
server_key_version: "encryptionkey@0"
I20260811 09:13:07.356794 1276 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439250192320-29290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439250192320-29290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439250192320-29290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:13:07.376219 1276 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:13:07.377126 1276 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:13:07.377411 1276 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:13:07.395365 1276 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.190:34443
I20260811 09:13:07.395398 1409 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.190:34443 every 8 connection(s)
I20260811 09:13:07.396770 1276 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
I20260811 09:13:07.400646 1411 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:13:07.404582 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 1276
I20260811 09:13:07.404750 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439250192320-29290-0/minicluster-data/master-0/wal/instance
I20260811 09:13:07.405189 29290 external_mini_cluster.cc:1428] Setting key 861970418506d502617a11b1dc8c2801
I20260811 09:13:07.413287 1411 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8cde04e09534d9887748b52b4357b16: Bootstrap starting.
I20260811 09:13:07.416973 1411 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e8cde04e09534d9887748b52b4357b16: Neither blocks nor log segments found. Creating new log.
I20260811 09:13:07.418239 1411 log.cc:824] T 00000000000000000000000000000000 P e8cde04e09534d9887748b52b4357b16: Log is configured to *not* fsync() on all Append() calls
I20260811 09:13:07.421419 1416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:07.407192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54708 (local address 127.28.154.190:34443)
0811 09:13:07.408276 (+ 1084us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:07.408291 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:07.408380 (+ 89us) server_negotiation.cc:407] Connection header received
0811 09:13:07.411317 (+ 2937us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:07.411350 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:07.411983 (+ 633us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:07.412453 (+ 470us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:07.413020 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:07.414981 (+ 1961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:07.416404 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:07.417461 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:07.417644 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:07.418696 (+ 1052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:13:07.418708 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:13:07.418729 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:13:07.419384 (+ 655us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:13:07.419465 (+ 81us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:13:07.419487 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:13:07.419594 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:07.419830 (+ 236us) server_negotiation.cc:299] Negotiation successful
0811 09:13:07.420168 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":587,"thread_start_us":335,"threads_started":1}
I20260811 09:13:07.421692 1411 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8cde04e09534d9887748b52b4357b16: No bootstrap required, opened a new log
I20260811 09:13:07.429646 1411 raft_consensus.cc:348] T 00000000000000000000000000000000 P e8cde04e09534d9887748b52b4357b16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8cde04e09534d9887748b52b4357b16" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 34443 } }
I20260811 09:13:07.430088 1411 raft_consensus.cc:374] T 00000000000000000000000000000000 P e8cde04e09534d9887748b52b4357b16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:13:07.430197 1411 raft_consensus.cc:729] T 00000000000000000000000000000000 P e8cde04e09534d9887748b52b4357b16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8cde04e09534d9887748b52b4357b16, State: Initialized, Role: FOLLOWER
I20260811 09:13:07.430786 1411 consensus_queue.cc:260] T 00000000000000000000000000000000 P e8cde04e09534d9887748b52b4357b16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8cde04e09534d9887748b52b4357b16" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 34443 } }
I20260811 09:13:07.431013 1411 raft_consensus.cc:388] T 00000000000000000000000000000000 P e8cde04e09534d9887748b52b4357b16 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:13:07.431150 1411 raft_consensus.cc:482] T 00000000000000000000000000000000 P e8cde04e09534d9887748b52b4357b16 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:13:07.431314 1411 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e8cde04e09534d9887748b52b4357b16 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:13:07.433573 1411 raft_consensus.cc:504] T 00000000000000000000000000000000 P e8cde04e09534d9887748b52b4357b16 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8cde04e09534d9887748b52b4357b16" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 34443 } }
I20260811 09:13:07.434069 1411 leader_election.cc:302] T 00000000000000000000000000000000 P e8cde04e09534d9887748b52b4357b16 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e8cde04e09534d9887748b52b4357b16; no voters:
I20260811 09:13:07.434700 1411 leader_election.cc:288] T 00000000000000000000000000000000 P e8cde04e09534d9887748b52b4357b16 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:13:07.434863 1420 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e8cde04e09534d9887748b52b4357b16 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:13:07.435920 1420 raft_consensus.cc:686] T 00000000000000000000000000000000 P e8cde04e09534d9887748b52b4357b16 [term 1 LEADER]: Becoming Leader. State: Replica: e8cde04e09534d9887748b52b4357b16, State: Running, Role: LEADER
I20260811 09:13:07.436632 1420 consensus_queue.cc:237] T 00000000000000000000000000000000 P e8cde04e09534d9887748b52b4357b16 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8cde04e09534d9887748b52b4357b16" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 34443 } }
I20260811 09:13:07.436933 1411 sys_catalog.cc:564] T 00000000000000000000000000000000 P e8cde04e09534d9887748b52b4357b16 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:13:07.439463 1421 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8cde04e09534d9887748b52b4357b16 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e8cde04e09534d9887748b52b4357b16" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8cde04e09534d9887748b52b4357b16" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 34443 } } }
I20260811 09:13:07.439662 1422 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8cde04e09534d9887748b52b4357b16 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e8cde04e09534d9887748b52b4357b16. Latest consensus state: current_term: 1 leader_uuid: "e8cde04e09534d9887748b52b4357b16" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8cde04e09534d9887748b52b4357b16" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 34443 } } }
I20260811 09:13:07.439909 1421 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8cde04e09534d9887748b52b4357b16 [sys.catalog]: This master's current role is: LEADER
I20260811 09:13:07.439958 1422 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8cde04e09534d9887748b52b4357b16 [sys.catalog]: This master's current role is: LEADER
I20260811 09:13:07.441658 1434 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:13:07.446132 1434 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:13:07.454759 1434 catalog_manager.cc:1347] Generated new cluster ID: 4cb5065eb999437f83aab632586273e3
I20260811 09:13:07.454887 1434 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:13:07.472168 1434 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:13:07.474112 1434 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:13:07.487504 1434 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e8cde04e09534d9887748b52b4357b16: Generated new TSK 0
I20260811 09:13:07.488435 1434 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 09:13:07.497117 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439250192320-29290-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439250192320-29290-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439250192320-29290-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439250192320-29290-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.154.129:0
--local_ip_for_outbound_sockets=127.28.154.129
--webserver_interface=127.28.154.129
--webserver_port=0
--tserver_master_addrs=127.28.154.190:34443
--builtin_ntp_servers=127.28.154.148:46103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 09:13:07.873836 1450 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:13:07.874152 1450 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:13:07.874267 1450 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:13:07.881676 1450 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:13:07.881789 1450 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:13:07.881930 1450 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.154.129
I20260811 09:13:07.891315 1450 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:46103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439250192320-29290-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439250192320-29290-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.154.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439250192320-29290-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.154.129
--webserver_port=0
--tserver_master_addrs=127.28.154.190:34443
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.154.129
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439250192320-29290-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:13:07.892731 1450 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:13:07.894527 1450 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:13:07.907613 1464 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:13:07.907554 1465 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:13:07.907536 1469 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:13:07.908439 1450 server_base.cc:1034] running on GCE node
I20260811 09:13:07.909044 1450 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:13:07.910004 1450 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:13:07.911226 1450 hybrid_clock.cc:648] HybridClock initialized: now 1786439587911169 us; error 45 us; skew 500 ppm
I20260811 09:13:07.911567 1450 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 09:13:07.913957 1450 webserver.cc:460] Webserver started at http://127.28.154.129:46807/ using document root <none> and password file <none>
I20260811 09:13:07.914654 1450 fs_manager.cc:359] Metadata directory not provided
I20260811 09:13:07.914745 1450 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:13:07.915033 1450 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:13:07.917917 1450 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439250192320-29290-0/minicluster-data/ts-0/data/instance:
uuid: "9a4a8c7e211e4c50b53c98f28b0da268"
format_stamp: "Formatted at 2026-08-11 09:13:07 on dist-test-slave-xt4k"
server_key: "2c39d23d44790dba41993ef7f923dc69"
server_key_iv: "1d979d9e5ddee8b8d81823a3b044e7a4"
server_key_version: "encryptionkey@0"
I20260811 09:13:07.918691 1450 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439250192320-29290-0/minicluster-data/ts-0/wal/instance:
uuid: "9a4a8c7e211e4c50b53c98f28b0da268"
format_stamp: "Formatted at 2026-08-11 09:13:07 on dist-test-slave-xt4k"
server_key: "2c39d23d44790dba41993ef7f923dc69"
server_key_iv: "1d979d9e5ddee8b8d81823a3b044e7a4"
server_key_version: "encryptionkey@0"
I20260811 09:13:07.924283 1450 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260811 09:13:07.928015 1480 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:13:07.929616 1450 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 09:13:07.929804 1450 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439250192320-29290-0/minicluster-data/ts-0/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439250192320-29290-0/minicluster-data/ts-0/wal
uuid: "9a4a8c7e211e4c50b53c98f28b0da268"
format_stamp: "Formatted at 2026-08-11 09:13:07 on dist-test-slave-xt4k"
server_key: "2c39d23d44790dba41993ef7f923dc69"
server_key_iv: "1d979d9e5ddee8b8d81823a3b044e7a4"
server_key_version: "encryptionkey@0"
I20260811 09:13:07.930020 1450 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439250192320-29290-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439250192320-29290-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439250192320-29290-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:13:07.962298 1450 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:13:07.963195 1450 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:13:07.963482 1450 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:13:07.964600 1450 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:13:07.966185 1450 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:13:07.966286 1450 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:13:07.966398 1450 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:13:07.966468 1450 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:13:07.998272 1450 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.129:42473
I20260811 09:13:07.998361 1697 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.129:42473 every 8 connection(s)
I20260811 09:13:07.999612 1450 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439250192320-29290-0/minicluster-data/ts-0/data/info.pb
I20260811 09:13:08.005081 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 1450
I20260811 09:13:08.005255 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439250192320-29290-0/minicluster-data/ts-0/wal/instance
I20260811 09:13:08.005730 29290 external_mini_cluster.cc:1428] Setting key 0613f8176e5327906bb314ddd309f643
I20260811 09:13:08.008792 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439250192320-29290-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439250192320-29290-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439250192320-29290-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439250192320-29290-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.154.130:0
--local_ip_for_outbound_sockets=127.28.154.130
--webserver_interface=127.28.154.130
--webserver_port=0
--tserver_master_addrs=127.28.154.190:34443
--builtin_ntp_servers=127.28.154.148:46103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 09:13:08.013795 1704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:08.002498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.129:44181 (local address 127.28.154.190:34443)
0811 09:13:08.003011 (+ 513us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:08.003018 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:08.004305 (+ 1287us) server_negotiation.cc:407] Connection header received
0811 09:13:08.006283 (+ 1978us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:08.006288 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:08.006359 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:08.006467 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:08.007573 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:08.008696 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:08.010615 (+ 1919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:08.011184 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:08.011332 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:08.012882 (+ 1550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:13:08.012888 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:13:08.012892 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:13:08.012918 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:13:08.012971 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:13:08.012987 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:13:08.013081 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:08.013340 (+ 259us) server_negotiation.cc:299] Negotiation successful
0811 09:13:08.013446 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":233,"threads_started":1}
I20260811 09:13:08.014886 1707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:08.003132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:34443 (local address 127.28.154.129:44181)
0811 09:13:08.004125 (+ 993us) negotiation.cc:107] Waiting for socket to connect
0811 09:13:08.004163 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 09:13:08.005979 (+ 1816us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:13:08.006693 (+ 714us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:13:08.006702 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:13:08.006793 (+ 91us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:13:08.007393 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:13:08.007422 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:13:08.008903 (+ 1481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:13:08.008910 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:13:08.010435 (+ 1525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:13:08.010450 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:13:08.011445 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:13:08.011451 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:13:08.011701 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:08.012497 (+ 796us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 09:13:08.012500 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:13:08.012562 (+ 62us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 09:13:08.012659 (+ 97us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:13:08.013114 (+ 455us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:13:08.013125 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:13:08.013133 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 09:13:08.013311 (+ 178us) client_negotiation.cc:240] Negotiation successful
0811 09:13:08.013679 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":530,"thread_start_us":324,"threads_started":1}
I20260811 09:13:08.016676 1699 heartbeater.cc:347] Connected to a master server at 127.28.154.190:34443
I20260811 09:13:08.017014 1699 heartbeater.cc:464] Registering TS with master...
I20260811 09:13:08.017796 1699 heartbeater.cc:511] Master 127.28.154.190:34443 requested a full tablet report, sending...
I20260811 09:13:08.020072 1338 ts_manager.cc:194] Registered new tserver with Master: 9a4a8c7e211e4c50b53c98f28b0da268 (127.28.154.129:42473)
I20260811 09:13:08.022055 1338 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.28.154.129:44181
W20260811 09:13:08.356168 1713 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:13:08.356493 1713 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:13:08.356630 1713 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:13:08.364101 1713 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:13:08.364199 1713 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:13:08.364353 1713 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.154.130
I20260811 09:13:08.373838 1713 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:46103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439250192320-29290-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439250192320-29290-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.154.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439250192320-29290-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.154.130
--webserver_port=0
--tserver_master_addrs=127.28.154.190:34443
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.154.130
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439250192320-29290-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:13:08.375154 1713 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:13:08.376977 1713 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:13:08.390369 1723 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:13:08.390447 1721 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:13:08.390571 1719 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:13:08.391294 1713 server_base.cc:1034] running on GCE node
I20260811 09:13:08.391978 1713 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:13:08.393009 1713 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:13:08.394230 1713 hybrid_clock.cc:648] HybridClock initialized: now 1786439588394174 us; error 51 us; skew 500 ppm
I20260811 09:13:08.394584 1713 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 09:13:08.397450 1713 webserver.cc:460] Webserver started at http://127.28.154.130:34421/ using document root <none> and password file <none>
I20260811 09:13:08.398311 1713 fs_manager.cc:359] Metadata directory not provided
I20260811 09:13:08.398433 1713 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:13:08.398814 1713 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:13:08.402583 1713 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439250192320-29290-0/minicluster-data/ts-1/data/instance:
uuid: "3ee520321cbb45a6b757dbedf5839520"
format_stamp: "Formatted at 2026-08-11 09:13:08 on dist-test-slave-xt4k"
server_key: "2cec0506f3dd26f20583837a7237aa50"
server_key_iv: "d2fe414f072b3a9c1717df38ae6bf77b"
server_key_version: "encryptionkey@0"
I20260811 09:13:08.403577 1713 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439250192320-29290-0/minicluster-data/ts-1/wal/instance:
uuid: "3ee520321cbb45a6b757dbedf5839520"
format_stamp: "Formatted at 2026-08-11 09:13:08 on dist-test-slave-xt4k"
server_key: "2cec0506f3dd26f20583837a7237aa50"
server_key_iv: "d2fe414f072b3a9c1717df38ae6bf77b"
server_key_version: "encryptionkey@0"
I20260811 09:13:08.409940 1713 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.003s
I20260811 09:13:08.414260 1732 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:13:08.415673 1713 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 09:13:08.415822 1713 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439250192320-29290-0/minicluster-data/ts-1/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439250192320-29290-0/minicluster-data/ts-1/wal
uuid: "3ee520321cbb45a6b757dbedf5839520"
format_stamp: "Formatted at 2026-08-11 09:13:08 on dist-test-slave-xt4k"
server_key: "2cec0506f3dd26f20583837a7237aa50"
server_key_iv: "d2fe414f072b3a9c1717df38ae6bf77b"
server_key_version: "encryptionkey@0"
I20260811 09:13:08.416018 1713 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439250192320-29290-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439250192320-29290-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439250192320-29290-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:13:08.435322 1713 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:13:08.436185 1713 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:13:08.436532 1713 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:13:08.437638 1713 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:13:08.439348 1713 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:13:08.439450 1713 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:13:08.439563 1713 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:13:08.439643 1713 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:13:08.471248 1713 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.130:39231
I20260811 09:13:08.471295 1955 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.130:39231 every 8 connection(s)
I20260811 09:13:08.472626 1713 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439250192320-29290-0/minicluster-data/ts-1/data/info.pb
I20260811 09:13:08.473520 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 1713
I20260811 09:13:08.473701 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439250192320-29290-0/minicluster-data/ts-1/wal/instance
I20260811 09:13:08.474192 29290 external_mini_cluster.cc:1428] Setting key 06c62f2cd9f70cd82fa9a950581d807a
I20260811 09:13:08.477466 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439250192320-29290-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439250192320-29290-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439250192320-29290-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439250192320-29290-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.154.131:0
--local_ip_for_outbound_sockets=127.28.154.131
--webserver_interface=127.28.154.131
--webserver_port=0
--tserver_master_addrs=127.28.154.190:34443
--builtin_ntp_servers=127.28.154.148:46103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 09:13:08.487047 1704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:08.475368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.130:49685 (local address 127.28.154.190:34443)
0811 09:13:08.475585 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:08.475592 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:08.477135 (+ 1543us) server_negotiation.cc:407] Connection header received
0811 09:13:08.479477 (+ 2342us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:08.479507 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:08.479575 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:08.479661 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:08.480999 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:08.481779 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:08.484013 (+ 2234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:08.484464 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:08.484596 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:08.486269 (+ 1673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:13:08.486275 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:13:08.486278 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:13:08.486299 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:13:08.486333 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:13:08.486346 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:13:08.486463 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:08.486713 (+ 250us) server_negotiation.cc:299] Negotiation successful
0811 09:13:08.486805 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 09:13:08.488387 1961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:08.475842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:34443 (local address 127.28.154.130:49685)
0811 09:13:08.476937 (+ 1095us) negotiation.cc:107] Waiting for socket to connect
0811 09:13:08.477017 (+ 80us) client_negotiation.cc:174] Beginning negotiation
0811 09:13:08.479177 (+ 2160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:13:08.479904 (+ 727us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:13:08.479915 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:13:08.480037 (+ 122us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:13:08.480818 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:13:08.480843 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:13:08.481944 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:13:08.481955 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:13:08.483860 (+ 1905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:13:08.483875 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:13:08.484719 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:13:08.484726 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:13:08.485016 (+ 290us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:08.485869 (+ 853us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 09:13:08.485872 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:13:08.485927 (+ 55us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 09:13:08.486052 (+ 125us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:13:08.486478 (+ 426us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:13:08.486489 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:13:08.486497 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 09:13:08.486679 (+ 182us) client_negotiation.cc:240] Negotiation successful
0811 09:13:08.487108 (+ 429us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":542,"thread_start_us":317,"threads_started":1}
I20260811 09:13:08.489845 1959 heartbeater.cc:347] Connected to a master server at 127.28.154.190:34443
I20260811 09:13:08.490149 1959 heartbeater.cc:464] Registering TS with master...
I20260811 09:13:08.490952 1959 heartbeater.cc:511] Master 127.28.154.190:34443 requested a full tablet report, sending...
I20260811 09:13:08.492677 1338 ts_manager.cc:194] Registered new tserver with Master: 3ee520321cbb45a6b757dbedf5839520 (127.28.154.130:39231)
I20260811 09:13:08.493526 1338 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.28.154.130:49685
W20260811 09:13:08.819793 1965 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:13:08.820081 1965 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:13:08.820207 1965 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:13:08.827522 1965 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:13:08.827636 1965 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:13:08.827780 1965 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.154.131
I20260811 09:13:08.837181 1965 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:46103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439250192320-29290-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439250192320-29290-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.154.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439250192320-29290-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.154.131
--webserver_port=0
--tserver_master_addrs=127.28.154.190:34443
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.154.131
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439250192320-29290-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:13:08.838562 1965 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:13:08.840325 1965 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:13:08.856801 1974 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:13:08.856875 1975 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:13:08.856829 1978 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:13:08.857564 1965 server_base.cc:1034] running on GCE node
I20260811 09:13:08.858167 1965 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:13:08.859077 1965 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:13:08.860373 1965 hybrid_clock.cc:648] HybridClock initialized: now 1786439588860286 us; error 47 us; skew 500 ppm
I20260811 09:13:08.860745 1965 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260811 09:13:08.863044 1965 webserver.cc:460] Webserver started at http://127.28.154.131:44361/ using document root <none> and password file <none>
I20260811 09:13:08.863710 1965 fs_manager.cc:359] Metadata directory not provided
I20260811 09:13:08.863794 1965 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:13:08.864046 1965 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:13:08.866892 1965 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439250192320-29290-0/minicluster-data/ts-2/data/instance:
uuid: "598c84edb47241acab625318195ffecf"
format_stamp: "Formatted at 2026-08-11 09:13:08 on dist-test-slave-xt4k"
server_key: "ce241ae4b3bf79f1938020b652986dc1"
server_key_iv: "6dab6aa62fc64d27d1fdcf01b7965af4"
server_key_version: "encryptionkey@0"
I20260811 09:13:08.867659 1965 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439250192320-29290-0/minicluster-data/ts-2/wal/instance:
uuid: "598c84edb47241acab625318195ffecf"
format_stamp: "Formatted at 2026-08-11 09:13:08 on dist-test-slave-xt4k"
server_key: "ce241ae4b3bf79f1938020b652986dc1"
server_key_iv: "6dab6aa62fc64d27d1fdcf01b7965af4"
server_key_version: "encryptionkey@0"
I20260811 09:13:08.872841 1965 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260811 09:13:08.876237 1987 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:13:08.877755 1965 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 09:13:08.877947 1965 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439250192320-29290-0/minicluster-data/ts-2/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439250192320-29290-0/minicluster-data/ts-2/wal
uuid: "598c84edb47241acab625318195ffecf"
format_stamp: "Formatted at 2026-08-11 09:13:08 on dist-test-slave-xt4k"
server_key: "ce241ae4b3bf79f1938020b652986dc1"
server_key_iv: "6dab6aa62fc64d27d1fdcf01b7965af4"
server_key_version: "encryptionkey@0"
I20260811 09:13:08.878165 1965 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439250192320-29290-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439250192320-29290-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439250192320-29290-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:13:08.910326 1965 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:13:08.911263 1965 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:13:08.911612 1965 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:13:08.912748 1965 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 09:13:08.914299 1965 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 09:13:08.914397 1965 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 09:13:08.914479 1965 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 09:13:08.914533 1965 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 09:13:08.946540 1965 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.131:33875
I20260811 09:13:08.946604 2194 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.131:33875 every 8 connection(s)
I20260811 09:13:08.947821 1965 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439250192320-29290-0/minicluster-data/ts-2/data/info.pb
I20260811 09:13:08.952507 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 1965
I20260811 09:13:08.952696 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439250192320-29290-0/minicluster-data/ts-2/wal/instance
I20260811 09:13:08.953119 29290 external_mini_cluster.cc:1428] Setting key e40e30ce999553dbb9aa0a9c78b247eb
I20260811 09:13:08.960970 1704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:08.950433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.131:35247 (local address 127.28.154.190:34443)
0811 09:13:08.950614 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:08.950619 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:08.952006 (+ 1387us) server_negotiation.cc:407] Connection header received
0811 09:13:08.954190 (+ 2184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:08.954195 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:08.954274 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:08.954389 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:08.955750 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:08.956707 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:08.958277 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:08.958711 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:08.958808 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:08.960094 (+ 1286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:13:08.960101 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:13:08.960106 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:13:08.960129 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:13:08.960163 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:13:08.960176 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:13:08.960245 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:08.960493 (+ 248us) server_negotiation.cc:299] Negotiation successful
0811 09:13:08.960623 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 09:13:08.961984 2202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:08.950877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:34443 (local address 127.28.154.131:35247)
0811 09:13:08.951809 (+ 932us) negotiation.cc:107] Waiting for socket to connect
0811 09:13:08.951852 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 09:13:08.953893 (+ 2041us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:13:08.954644 (+ 751us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:13:08.954653 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:13:08.954753 (+ 100us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:13:08.955573 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:13:08.955601 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:13:08.956850 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:13:08.956854 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:13:08.958131 (+ 1277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:13:08.958149 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:13:08.958905 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:13:08.958910 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:13:08.959129 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:08.959774 (+ 645us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 09:13:08.959777 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:13:08.959818 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 09:13:08.959901 (+ 83us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:13:08.960248 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:13:08.960255 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:13:08.960260 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 09:13:08.960448 (+ 188us) client_negotiation.cc:240] Negotiation successful
0811 09:13:08.960896 (+ 448us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":286,"threads_started":1}
I20260811 09:13:08.963544 2196 heartbeater.cc:347] Connected to a master server at 127.28.154.190:34443
I20260811 09:13:08.963836 2196 heartbeater.cc:464] Registering TS with master...
I20260811 09:13:08.964604 2196 heartbeater.cc:511] Master 127.28.154.190:34443 requested a full tablet report, sending...
I20260811 09:13:08.966125 1338 ts_manager.cc:194] Registered new tserver with Master: 598c84edb47241acab625318195ffecf (127.28.154.131:33875)
I20260811 09:13:08.967041 1338 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.28.154.131:35247
I20260811 09:13:08.968552 29290 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 09:13:08.978616 1704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:08.972927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54716 (local address 127.28.154.190:34443)
0811 09:13:08.973112 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:08.973119 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:08.973467 (+ 348us) server_negotiation.cc:407] Connection header received
0811 09:13:08.973641 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:08.973647 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:08.973717 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:08.973801 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:08.974260 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:08.974941 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:08.976272 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:08.976782 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:08.976896 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:08.977923 (+ 1027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:13:08.977930 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:13:08.977933 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:13:08.977952 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:13:08.977986 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:13:08.977998 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:13:08.978082 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:08.978202 (+ 120us) server_negotiation.cc:299] Negotiation successful
0811 09:13:08.978305 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 09:13:08.982663 1338 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54716:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 09:13:08.989701 1338 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 09:13:09.016948 2229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:09.003005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.131:33875 (local address 127.0.0.1:39014)
0811 09:13:09.004598 (+ 1593us) negotiation.cc:107] Waiting for socket to connect
0811 09:13:09.004639 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 09:13:09.004826 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:13:09.007119 (+ 2293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:13:09.007129 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:13:09.007166 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:13:09.007649 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:13:09.007663 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:13:09.010288 (+ 2625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:13:09.010295 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:13:09.013313 (+ 3018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:13:09.013339 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:13:09.015859 (+ 2520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:13:09.015868 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:13:09.016140 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:09.016211 (+ 71us) client_negotiation.cc:769] Sending connection context
0811 09:13:09.016359 (+ 148us) client_negotiation.cc:240] Negotiation successful
0811 09:13:09.016599 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1340,"thread_start_us":2094,"threads_started":1}
I20260811 09:13:09.017424 2230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:09.003271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39014 (local address 127.28.154.131:33875)
0811 09:13:09.006335 (+ 3064us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:09.006345 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:09.006387 (+ 42us) server_negotiation.cc:407] Connection header received
0811 09:13:09.006504 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:09.006513 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:09.006815 (+ 302us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:09.007922 (+ 1107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:13:09.008255 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:09.010075 (+ 1820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:09.013765 (+ 3690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:09.015521 (+ 1756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:09.016547 (+ 1026us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:09.016812 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:09.016899 (+ 87us) server_negotiation.cc:299] Negotiation successful
0811 09:13:09.017103 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2856,"thread_start_us":2684,"threads_started":1}
I20260811 09:13:09.022094 2069 tablet_service.cc:1467] Processing CreateTablet for tablet 18401d9beffe4bb79279b04a0408f596 (DEFAULT_TABLE table=test-table [id=d177e1d95f8f411096bfd02ea0a230d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:13:09.023895 2069 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 18401d9beffe4bb79279b04a0408f596. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:13:09.025875 1699 heartbeater.cc:503] Master 127.28.154.190:34443 was elected leader, sending a full tablet report...
I20260811 09:13:09.026342 2229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:09.006803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.129:42473 (local address 127.0.0.1:36968)
0811 09:13:09.017818 (+ 11015us) negotiation.cc:107] Waiting for socket to connect
0811 09:13:09.017840 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 09:13:09.017938 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:13:09.019896 (+ 1958us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:13:09.019902 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:13:09.019933 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:13:09.020136 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:13:09.020147 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:13:09.021869 (+ 1722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:13:09.021875 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:13:09.023480 (+ 1605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:13:09.023504 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:13:09.025407 (+ 1903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:13:09.025414 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:13:09.025636 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:09.025674 (+ 38us) client_negotiation.cc:769] Sending connection context
0811 09:13:09.025785 (+ 111us) client_negotiation.cc:240] Negotiation successful
0811 09:13:09.025986 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10559,"mutex_wait_us":5,"thread_start_us":12464,"threads_started":1}
I20260811 09:13:09.026888 2228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:09.003141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.130:39231 (local address 127.0.0.1:60032)
0811 09:13:09.005404 (+ 2263us) negotiation.cc:107] Waiting for socket to connect
0811 09:13:09.005422 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 09:13:09.005652 (+ 230us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:13:09.017726 (+ 12074us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:13:09.017732 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:13:09.017768 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:13:09.019111 (+ 1343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:13:09.019122 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:13:09.021166 (+ 2044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:13:09.021171 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:13:09.023389 (+ 2218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:13:09.023407 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:13:09.025775 (+ 2368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:13:09.025782 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:13:09.026042 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:09.026075 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 09:13:09.026171 (+ 96us) client_negotiation.cc:240] Negotiation successful
0811 09:13:09.026314 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2120,"mutex_wait_us":60,"thread_start_us":1016,"threads_started":1}
I20260811 09:13:09.027120 2232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:09.003653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60032 (local address 127.28.154.130:39231)
0811 09:13:09.016910 (+ 13257us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:09.016920 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:09.016957 (+ 37us) server_negotiation.cc:407] Connection header received
0811 09:13:09.017067 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:09.017076 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:09.017394 (+ 318us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:09.017658 (+ 264us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:13:09.019294 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:09.021019 (+ 1725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:09.023638 (+ 2619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:09.025191 (+ 1553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:09.025371 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:09.025608 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:09.026459 (+ 851us) server_negotiation.cc:299] Negotiation successful
0811 09:13:09.026732 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12570,"thread_start_us":12442,"threads_started":1}
I20260811 09:13:09.029385 2236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:09.007149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36968 (local address 127.28.154.129:42473)
0811 09:13:09.018475 (+ 11326us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:09.018495 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:09.018526 (+ 31us) server_negotiation.cc:407] Connection header received
0811 09:13:09.018641 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:09.018649 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:09.019683 (+ 1034us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:09.019912 (+ 229us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:13:09.020287 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:09.021687 (+ 1400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:09.023693 (+ 2006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:09.025191 (+ 1498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:09.025403 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:09.025608 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:09.027487 (+ 1879us) server_negotiation.cc:299] Negotiation successful
0811 09:13:09.029001 (+ 1514us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11141,"thread_start_us":11018,"threads_started":1}
I20260811 09:13:09.031332 1826 tablet_service.cc:1467] Processing CreateTablet for tablet 18401d9beffe4bb79279b04a0408f596 (DEFAULT_TABLE table=test-table [id=d177e1d95f8f411096bfd02ea0a230d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:13:09.032645 1826 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 18401d9beffe4bb79279b04a0408f596. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:13:09.033252 1575 tablet_service.cc:1467] Processing CreateTablet for tablet 18401d9beffe4bb79279b04a0408f596 (DEFAULT_TABLE table=test-table [id=d177e1d95f8f411096bfd02ea0a230d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 09:13:09.034754 1575 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 18401d9beffe4bb79279b04a0408f596. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:13:09.038724 2237 tablet_bootstrap.cc:492] T 18401d9beffe4bb79279b04a0408f596 P 598c84edb47241acab625318195ffecf: Bootstrap starting.
I20260811 09:13:09.042572 2237 tablet_bootstrap.cc:654] T 18401d9beffe4bb79279b04a0408f596 P 598c84edb47241acab625318195ffecf: Neither blocks nor log segments found. Creating new log.
I20260811 09:13:09.043831 2237 log.cc:824] T 18401d9beffe4bb79279b04a0408f596 P 598c84edb47241acab625318195ffecf: Log is configured to *not* fsync() on all Append() calls
I20260811 09:13:09.046636 2240 tablet_bootstrap.cc:492] T 18401d9beffe4bb79279b04a0408f596 P 3ee520321cbb45a6b757dbedf5839520: Bootstrap starting.
I20260811 09:13:09.049837 2241 tablet_bootstrap.cc:492] T 18401d9beffe4bb79279b04a0408f596 P 9a4a8c7e211e4c50b53c98f28b0da268: Bootstrap starting.
I20260811 09:13:09.050612 2240 tablet_bootstrap.cc:654] T 18401d9beffe4bb79279b04a0408f596 P 3ee520321cbb45a6b757dbedf5839520: Neither blocks nor log segments found. Creating new log.
I20260811 09:13:09.051506 2237 tablet_bootstrap.cc:492] T 18401d9beffe4bb79279b04a0408f596 P 598c84edb47241acab625318195ffecf: No bootstrap required, opened a new log
I20260811 09:13:09.051570 2240 log.cc:824] T 18401d9beffe4bb79279b04a0408f596 P 3ee520321cbb45a6b757dbedf5839520: Log is configured to *not* fsync() on all Append() calls
I20260811 09:13:09.051805 2237 ts_tablet_manager.cc:1397] T 18401d9beffe4bb79279b04a0408f596 P 598c84edb47241acab625318195ffecf: Time spent bootstrapping tablet: real 0.013s user 0.003s sys 0.008s
I20260811 09:13:09.053470 2241 tablet_bootstrap.cc:654] T 18401d9beffe4bb79279b04a0408f596 P 9a4a8c7e211e4c50b53c98f28b0da268: Neither blocks nor log segments found. Creating new log.
I20260811 09:13:09.054104 2240 tablet_bootstrap.cc:492] T 18401d9beffe4bb79279b04a0408f596 P 3ee520321cbb45a6b757dbedf5839520: No bootstrap required, opened a new log
I20260811 09:13:09.054352 2240 ts_tablet_manager.cc:1397] T 18401d9beffe4bb79279b04a0408f596 P 3ee520321cbb45a6b757dbedf5839520: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260811 09:13:09.054731 2241 log.cc:824] T 18401d9beffe4bb79279b04a0408f596 P 9a4a8c7e211e4c50b53c98f28b0da268: Log is configured to *not* fsync() on all Append() calls
I20260811 09:13:09.062731 2240 raft_consensus.cc:348] T 18401d9beffe4bb79279b04a0408f596 P 3ee520321cbb45a6b757dbedf5839520 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "598c84edb47241acab625318195ffecf" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 33875 } } peers { permanent_uuid: "3ee520321cbb45a6b757dbedf5839520" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 39231 } } peers { permanent_uuid: "9a4a8c7e211e4c50b53c98f28b0da268" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 42473 } }
I20260811 09:13:09.062752 2237 raft_consensus.cc:348] T 18401d9beffe4bb79279b04a0408f596 P 598c84edb47241acab625318195ffecf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "598c84edb47241acab625318195ffecf" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 33875 } } peers { permanent_uuid: "3ee520321cbb45a6b757dbedf5839520" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 39231 } } peers { permanent_uuid: "9a4a8c7e211e4c50b53c98f28b0da268" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 42473 } }
I20260811 09:13:09.063241 2241 tablet_bootstrap.cc:492] T 18401d9beffe4bb79279b04a0408f596 P 9a4a8c7e211e4c50b53c98f28b0da268: No bootstrap required, opened a new log
I20260811 09:13:09.063140 2240 raft_consensus.cc:374] T 18401d9beffe4bb79279b04a0408f596 P 3ee520321cbb45a6b757dbedf5839520 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:13:09.063287 2237 raft_consensus.cc:374] T 18401d9beffe4bb79279b04a0408f596 P 598c84edb47241acab625318195ffecf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:13:09.063388 2237 raft_consensus.cc:729] T 18401d9beffe4bb79279b04a0408f596 P 598c84edb47241acab625318195ffecf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 598c84edb47241acab625318195ffecf, State: Initialized, Role: FOLLOWER
I20260811 09:13:09.063436 2240 raft_consensus.cc:729] T 18401d9beffe4bb79279b04a0408f596 P 3ee520321cbb45a6b757dbedf5839520 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ee520321cbb45a6b757dbedf5839520, State: Initialized, Role: FOLLOWER
I20260811 09:13:09.063623 2241 ts_tablet_manager.cc:1397] T 18401d9beffe4bb79279b04a0408f596 P 9a4a8c7e211e4c50b53c98f28b0da268: Time spent bootstrapping tablet: real 0.014s user 0.006s sys 0.007s
I20260811 09:13:09.064152 2240 consensus_queue.cc:260] T 18401d9beffe4bb79279b04a0408f596 P 3ee520321cbb45a6b757dbedf5839520 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "598c84edb47241acab625318195ffecf" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 33875 } } peers { permanent_uuid: "3ee520321cbb45a6b757dbedf5839520" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 39231 } } peers { permanent_uuid: "9a4a8c7e211e4c50b53c98f28b0da268" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 42473 } }
I20260811 09:13:09.064122 2237 consensus_queue.cc:260] T 18401d9beffe4bb79279b04a0408f596 P 598c84edb47241acab625318195ffecf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "598c84edb47241acab625318195ffecf" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 33875 } } peers { permanent_uuid: "3ee520321cbb45a6b757dbedf5839520" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 39231 } } peers { permanent_uuid: "9a4a8c7e211e4c50b53c98f28b0da268" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 42473 } }
I20260811 09:13:09.065423 2196 heartbeater.cc:503] Master 127.28.154.190:34443 was elected leader, sending a full tablet report...
I20260811 09:13:09.065858 1959 heartbeater.cc:503] Master 127.28.154.190:34443 was elected leader, sending a full tablet report...
I20260811 09:13:09.066619 2240 ts_tablet_manager.cc:1428] T 18401d9beffe4bb79279b04a0408f596 P 3ee520321cbb45a6b757dbedf5839520: Time spent starting tablet: real 0.012s user 0.008s sys 0.005s
I20260811 09:13:09.068022 2237 ts_tablet_manager.cc:1428] T 18401d9beffe4bb79279b04a0408f596 P 598c84edb47241acab625318195ffecf: Time spent starting tablet: real 0.016s user 0.004s sys 0.011s
I20260811 09:13:09.073621 2241 raft_consensus.cc:348] T 18401d9beffe4bb79279b04a0408f596 P 9a4a8c7e211e4c50b53c98f28b0da268 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "598c84edb47241acab625318195ffecf" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 33875 } } peers { permanent_uuid: "3ee520321cbb45a6b757dbedf5839520" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 39231 } } peers { permanent_uuid: "9a4a8c7e211e4c50b53c98f28b0da268" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 42473 } }
I20260811 09:13:09.074023 2241 raft_consensus.cc:374] T 18401d9beffe4bb79279b04a0408f596 P 9a4a8c7e211e4c50b53c98f28b0da268 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:13:09.074100 2241 raft_consensus.cc:729] T 18401d9beffe4bb79279b04a0408f596 P 9a4a8c7e211e4c50b53c98f28b0da268 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a4a8c7e211e4c50b53c98f28b0da268, State: Initialized, Role: FOLLOWER
I20260811 09:13:09.074752 2241 consensus_queue.cc:260] T 18401d9beffe4bb79279b04a0408f596 P 9a4a8c7e211e4c50b53c98f28b0da268 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "598c84edb47241acab625318195ffecf" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 33875 } } peers { permanent_uuid: "3ee520321cbb45a6b757dbedf5839520" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 39231 } } peers { permanent_uuid: "9a4a8c7e211e4c50b53c98f28b0da268" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 42473 } }
I20260811 09:13:09.076596 2241 ts_tablet_manager.cc:1428] T 18401d9beffe4bb79279b04a0408f596 P 9a4a8c7e211e4c50b53c98f28b0da268: Time spent starting tablet: real 0.013s user 0.009s sys 0.003s
W20260811 09:13:09.199683 2199 tablet.cc:2369] T 18401d9beffe4bb79279b04a0408f596 P 598c84edb47241acab625318195ffecf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 09:13:09.217686 2246 raft_consensus.cc:482] T 18401d9beffe4bb79279b04a0408f596 P 3ee520321cbb45a6b757dbedf5839520 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 09:13:09.218029 2246 raft_consensus.cc:504] T 18401d9beffe4bb79279b04a0408f596 P 3ee520321cbb45a6b757dbedf5839520 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "598c84edb47241acab625318195ffecf" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 33875 } } peers { permanent_uuid: "3ee520321cbb45a6b757dbedf5839520" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 39231 } } peers { permanent_uuid: "9a4a8c7e211e4c50b53c98f28b0da268" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 42473 } }
I20260811 09:13:09.219906 2246 leader_election.cc:288] T 18401d9beffe4bb79279b04a0408f596 P 3ee520321cbb45a6b757dbedf5839520 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 598c84edb47241acab625318195ffecf (127.28.154.131:33875), 9a4a8c7e211e4c50b53c98f28b0da268 (127.28.154.129:42473)
W20260811 09:13:09.224946 1960 tablet.cc:2369] T 18401d9beffe4bb79279b04a0408f596 P 3ee520321cbb45a6b757dbedf5839520: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 09:13:09.229260 2236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:09.220368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.130:48217 (local address 127.28.154.129:42473)
0811 09:13:09.220572 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:09.220578 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:09.221290 (+ 712us) server_negotiation.cc:407] Connection header received
0811 09:13:09.221515 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:09.221520 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:09.221599 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:09.221707 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:13:09.222463 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:09.223562 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:09.226819 (+ 3257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:09.228087 (+ 1268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:09.228260 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:09.228359 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:09.228859 (+ 500us) server_negotiation.cc:299] Negotiation successful
0811 09:13:09.229003 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 09:13:09.229260 2230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:09.220376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.154.130:43991 (local address 127.28.154.131:33875)
0811 09:13:09.220572 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:09.220578 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:09.221369 (+ 791us) server_negotiation.cc:407] Connection header received
0811 09:13:09.221502 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:09.221507 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:09.221618 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:09.221713 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 09:13:09.222502 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:09.223570 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:09.226821 (+ 3251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:09.228070 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:09.228260 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:09.228359 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:09.228859 (+ 500us) server_negotiation.cc:299] Negotiation successful
0811 09:13:09.229003 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 09:13:09.229450 2253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:09.220311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.129:42473 (local address 127.28.154.130:48217)
0811 09:13:09.221165 (+ 854us) negotiation.cc:107] Waiting for socket to connect
0811 09:13:09.221186 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 09:13:09.221362 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:13:09.221730 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:13:09.221736 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:13:09.221771 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:13:09.222287 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:13:09.222305 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:13:09.223723 (+ 1418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:13:09.223728 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:13:09.226552 (+ 2824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:13:09.226570 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:13:09.228382 (+ 1812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:13:09.228389 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:13:09.228606 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:09.228715 (+ 109us) client_negotiation.cc:769] Sending connection context
0811 09:13:09.228789 (+ 74us) client_negotiation.cc:240] Negotiation successful
0811 09:13:09.228988 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":685,"mutex_wait_us":312,"spinlock_wait_cycles":1280,"thread_start_us":441,"threads_started":1}
I20260811 09:13:09.229480 2258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:09.220418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.131:33875 (local address 127.28.154.130:43991)
0811 09:13:09.221268 (+ 850us) negotiation.cc:107] Waiting for socket to connect
0811 09:13:09.221288 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 09:13:09.221382 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:13:09.221731 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:13:09.221736 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:13:09.221777 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 09:13:09.222342 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:13:09.222352 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:13:09.223725 (+ 1373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:13:09.223729 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:13:09.226552 (+ 2823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:13:09.226570 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:13:09.228320 (+ 1750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:13:09.228328 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:13:09.228569 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:09.228670 (+ 101us) client_negotiation.cc:769] Sending connection context
0811 09:13:09.228789 (+ 119us) client_negotiation.cc:240] Negotiation successful
0811 09:13:09.228997 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":731,"mutex_wait_us":267,"spinlock_wait_cycles":3456,"thread_start_us":423,"threads_started":1}
I20260811 09:13:09.230906 2111 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18401d9beffe4bb79279b04a0408f596" candidate_uuid: "3ee520321cbb45a6b757dbedf5839520" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "598c84edb47241acab625318195ffecf" is_pre_election: true
I20260811 09:13:09.230906 1611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18401d9beffe4bb79279b04a0408f596" candidate_uuid: "3ee520321cbb45a6b757dbedf5839520" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a4a8c7e211e4c50b53c98f28b0da268" is_pre_election: true
I20260811 09:13:09.231523 2111 raft_consensus.cc:2454] T 18401d9beffe4bb79279b04a0408f596 P 598c84edb47241acab625318195ffecf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ee520321cbb45a6b757dbedf5839520 in term 0.
I20260811 09:13:09.231523 1611 raft_consensus.cc:2454] T 18401d9beffe4bb79279b04a0408f596 P 9a4a8c7e211e4c50b53c98f28b0da268 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ee520321cbb45a6b757dbedf5839520 in term 0.
I20260811 09:13:09.232286 1735 leader_election.cc:302] T 18401d9beffe4bb79279b04a0408f596 P 3ee520321cbb45a6b757dbedf5839520 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ee520321cbb45a6b757dbedf5839520, 9a4a8c7e211e4c50b53c98f28b0da268; no voters:
I20260811 09:13:09.232730 2246 raft_consensus.cc:2781] T 18401d9beffe4bb79279b04a0408f596 P 3ee520321cbb45a6b757dbedf5839520 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 09:13:09.232877 2246 raft_consensus.cc:482] T 18401d9beffe4bb79279b04a0408f596 P 3ee520321cbb45a6b757dbedf5839520 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 09:13:09.232973 2246 raft_consensus.cc:3037] T 18401d9beffe4bb79279b04a0408f596 P 3ee520321cbb45a6b757dbedf5839520 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:13:09.235271 2246 raft_consensus.cc:504] T 18401d9beffe4bb79279b04a0408f596 P 3ee520321cbb45a6b757dbedf5839520 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "598c84edb47241acab625318195ffecf" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 33875 } } peers { permanent_uuid: "3ee520321cbb45a6b757dbedf5839520" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 39231 } } peers { permanent_uuid: "9a4a8c7e211e4c50b53c98f28b0da268" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 42473 } }
I20260811 09:13:09.235857 2246 leader_election.cc:288] T 18401d9beffe4bb79279b04a0408f596 P 3ee520321cbb45a6b757dbedf5839520 [CANDIDATE]: Term 1 election: Requested vote from peers 598c84edb47241acab625318195ffecf (127.28.154.131:33875), 9a4a8c7e211e4c50b53c98f28b0da268 (127.28.154.129:42473)
I20260811 09:13:09.236428 2111 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18401d9beffe4bb79279b04a0408f596" candidate_uuid: "3ee520321cbb45a6b757dbedf5839520" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "598c84edb47241acab625318195ffecf"
I20260811 09:13:09.236428 1611 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18401d9beffe4bb79279b04a0408f596" candidate_uuid: "3ee520321cbb45a6b757dbedf5839520" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a4a8c7e211e4c50b53c98f28b0da268"
I20260811 09:13:09.236673 2111 raft_consensus.cc:3037] T 18401d9beffe4bb79279b04a0408f596 P 598c84edb47241acab625318195ffecf [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:13:09.236673 1611 raft_consensus.cc:3037] T 18401d9beffe4bb79279b04a0408f596 P 9a4a8c7e211e4c50b53c98f28b0da268 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:13:09.239918 2111 raft_consensus.cc:2454] T 18401d9beffe4bb79279b04a0408f596 P 598c84edb47241acab625318195ffecf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ee520321cbb45a6b757dbedf5839520 in term 1.
I20260811 09:13:09.239918 1611 raft_consensus.cc:2454] T 18401d9beffe4bb79279b04a0408f596 P 9a4a8c7e211e4c50b53c98f28b0da268 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ee520321cbb45a6b757dbedf5839520 in term 1.
I20260811 09:13:09.240640 1734 leader_election.cc:302] T 18401d9beffe4bb79279b04a0408f596 P 3ee520321cbb45a6b757dbedf5839520 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 3ee520321cbb45a6b757dbedf5839520, 598c84edb47241acab625318195ffecf, 9a4a8c7e211e4c50b53c98f28b0da268; no voters:
I20260811 09:13:09.241158 2246 raft_consensus.cc:2781] T 18401d9beffe4bb79279b04a0408f596 P 3ee520321cbb45a6b757dbedf5839520 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:13:09.241703 2246 raft_consensus.cc:686] T 18401d9beffe4bb79279b04a0408f596 P 3ee520321cbb45a6b757dbedf5839520 [term 1 LEADER]: Becoming Leader. State: Replica: 3ee520321cbb45a6b757dbedf5839520, State: Running, Role: LEADER
I20260811 09:13:09.242250 2246 consensus_queue.cc:237] T 18401d9beffe4bb79279b04a0408f596 P 3ee520321cbb45a6b757dbedf5839520 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "598c84edb47241acab625318195ffecf" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 33875 } } peers { permanent_uuid: "3ee520321cbb45a6b757dbedf5839520" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 39231 } } peers { permanent_uuid: "9a4a8c7e211e4c50b53c98f28b0da268" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 42473 } }
I20260811 09:13:09.247267 1335 catalog_manager.cc:5495] T 18401d9beffe4bb79279b04a0408f596 P 3ee520321cbb45a6b757dbedf5839520 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ee520321cbb45a6b757dbedf5839520 (127.28.154.130). New cstate: current_term: 1 leader_uuid: "3ee520321cbb45a6b757dbedf5839520" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "598c84edb47241acab625318195ffecf" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 33875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ee520321cbb45a6b757dbedf5839520" member_type: VOTER last_known_addr { host: "127.28.154.130" port: 39231 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9a4a8c7e211e4c50b53c98f28b0da268" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 42473 } health_report { overall_health: UNKNOWN } } }
W20260811 09:13:09.252013 1701 tablet.cc:2369] T 18401d9beffe4bb79279b04a0408f596 P 9a4a8c7e211e4c50b53c98f28b0da268: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 09:13:09.314939 2232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:09.309380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60034 (local address 127.28.154.130:39231)
0811 09:13:09.309577 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:09.309585 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:09.309693 (+ 108us) server_negotiation.cc:407] Connection header received
0811 09:13:09.309918 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:09.309923 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:09.310017 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:09.310178 (+ 161us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:13:09.310565 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:09.311286 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:09.312684 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:09.313125 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:09.313238 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:09.313767 (+ 529us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:13:09.314313 (+ 546us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:13:09.314459 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:09.314568 (+ 109us) server_negotiation.cc:299] Negotiation successful
0811 09:13:09.314656 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 09:13:09.323601 2111 raft_consensus.cc:1261] T 18401d9beffe4bb79279b04a0408f596 P 598c84edb47241acab625318195ffecf [term 1 FOLLOWER]: Refusing update from remote peer 3ee520321cbb45a6b757dbedf5839520: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:13:09.323603 1611 raft_consensus.cc:1261] T 18401d9beffe4bb79279b04a0408f596 P 9a4a8c7e211e4c50b53c98f28b0da268 [term 1 FOLLOWER]: Refusing update from remote peer 3ee520321cbb45a6b757dbedf5839520: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:13:09.324690 2259 consensus_queue.cc:1035] T 18401d9beffe4bb79279b04a0408f596 P 3ee520321cbb45a6b757dbedf5839520 [LEADER]: Connected to new peer: Peer: permanent_uuid: "598c84edb47241acab625318195ffecf" member_type: VOTER last_known_addr { host: "127.28.154.131" port: 33875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:13:09.325080 2246 consensus_queue.cc:1035] T 18401d9beffe4bb79279b04a0408f596 P 3ee520321cbb45a6b757dbedf5839520 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a4a8c7e211e4c50b53c98f28b0da268" member_type: VOTER last_known_addr { host: "127.28.154.129" port: 42473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:13:09.353577 1335 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:54716:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:13:09.353871 1335 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:54716:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 09:13:09.357594 1335 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e8cde04e09534d9887748b52b4357b16: Sending DeleteTablet for 3 replicas of tablet 18401d9beffe4bb79279b04a0408f596
I20260811 09:13:09.358963 2069 tablet_service.cc:1514] Processing DeleteTablet for tablet 18401d9beffe4bb79279b04a0408f596 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:13:09 UTC) from {username='slave'} at 127.0.0.1:39014
I20260811 09:13:09.359187 1575 tablet_service.cc:1514] Processing DeleteTablet for tablet 18401d9beffe4bb79279b04a0408f596 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:13:09 UTC) from {username='slave'} at 127.0.0.1:36968
I20260811 09:13:09.359746 1826 tablet_service.cc:1514] Processing DeleteTablet for tablet 18401d9beffe4bb79279b04a0408f596 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 09:13:09 UTC) from {username='slave'} at 127.0.0.1:60032
I20260811 09:13:09.359947 2282 tablet_replica.cc:330] T 18401d9beffe4bb79279b04a0408f596 P 598c84edb47241acab625318195ffecf: stopping tablet replica
I20260811 09:13:09.360489 2282 raft_consensus.cc:2229] T 18401d9beffe4bb79279b04a0408f596 P 598c84edb47241acab625318195ffecf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:13:09.361088 2282 raft_consensus.cc:2258] T 18401d9beffe4bb79279b04a0408f596 P 598c84edb47241acab625318195ffecf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:13:09.361886 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 1450
I20260811 09:13:09.367619 2282 ts_tablet_manager.cc:1905] T 18401d9beffe4bb79279b04a0408f596 P 598c84edb47241acab625318195ffecf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:13:09.373384 2283 tablet_replica.cc:330] T 18401d9beffe4bb79279b04a0408f596 P 9a4a8c7e211e4c50b53c98f28b0da268: stopping tablet replica
I20260811 09:13:09.373940 2283 raft_consensus.cc:2229] T 18401d9beffe4bb79279b04a0408f596 P 9a4a8c7e211e4c50b53c98f28b0da268 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 09:13:09.374565 2283 raft_consensus.cc:2258] T 18401d9beffe4bb79279b04a0408f596 P 9a4a8c7e211e4c50b53c98f28b0da268 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:13:09.375711 2284 tablet_replica.cc:330] T 18401d9beffe4bb79279b04a0408f596 P 3ee520321cbb45a6b757dbedf5839520: stopping tablet replica
I20260811 09:13:09.376156 2284 raft_consensus.cc:2229] T 18401d9beffe4bb79279b04a0408f596 P 3ee520321cbb45a6b757dbedf5839520 [term 1 LEADER]: Raft consensus shutting down.
I20260811 09:13:09.376853 2284 raft_consensus.cc:2258] T 18401d9beffe4bb79279b04a0408f596 P 3ee520321cbb45a6b757dbedf5839520 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 09:13:09.377238 2282 ts_tablet_manager.cc:1918] T 18401d9beffe4bb79279b04a0408f596 P 598c84edb47241acab625318195ffecf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:13:09.377420 2282 log.cc:1196] T 18401d9beffe4bb79279b04a0408f596 P 598c84edb47241acab625318195ffecf: Deleting WAL directory at /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439250192320-29290-0/minicluster-data/ts-2/wal/wals/18401d9beffe4bb79279b04a0408f596
I20260811 09:13:09.378331 2282 ts_tablet_manager.cc:1939] T 18401d9beffe4bb79279b04a0408f596 P 598c84edb47241acab625318195ffecf: Deleting consensus metadata
I20260811 09:13:09.379581 2284 ts_tablet_manager.cc:1905] T 18401d9beffe4bb79279b04a0408f596 P 3ee520321cbb45a6b757dbedf5839520: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:13:09.380479 2236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:09.363198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36982 (local address 127.28.154.129:42473)
0811 09:13:09.363388 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:09.363395 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:09.363421 (+ 26us) server_negotiation.cc:407] Connection header received
0811 09:13:09.363693 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:09.363699 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:09.363829 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:09.363914 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:09.364654 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:09.365906 (+ 1252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:09.375786 (+ 9880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:09.376485 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:09.376618 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:09.378459 (+ 1841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:13:09.378470 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:13:09.378489 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:13:09.379256 (+ 767us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:13:09.379321 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:13:09.379349 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:13:09.379451 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:09.379898 (+ 447us) server_negotiation.cc:299] Negotiation successful
0811 09:13:09.380052 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 09:13:09.381434 2283 ts_tablet_manager.cc:1905] T 18401d9beffe4bb79279b04a0408f596 P 9a4a8c7e211e4c50b53c98f28b0da268: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 09:13:09.381510 1311 catalog_manager.cc:4841] TS 598c84edb47241acab625318195ffecf (127.28.154.131:33875): tablet 18401d9beffe4bb79279b04a0408f596 (table test-table [id=d177e1d95f8f411096bfd02ea0a230d2]) successfully deleted
I20260811 09:13:09.383507 1690 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:13:09.387827 2284 ts_tablet_manager.cc:1918] T 18401d9beffe4bb79279b04a0408f596 P 3ee520321cbb45a6b757dbedf5839520: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 09:13:09.388016 2284 log.cc:1196] T 18401d9beffe4bb79279b04a0408f596 P 3ee520321cbb45a6b757dbedf5839520: Deleting WAL directory at /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786439250192320-29290-0/minicluster-data/ts-1/wal/wals/18401d9beffe4bb79279b04a0408f596
I20260811 09:13:09.388608 2284 ts_tablet_manager.cc:1939] T 18401d9beffe4bb79279b04a0408f596 P 3ee520321cbb45a6b757dbedf5839520: Deleting consensus metadata
I20260811 09:13:09.390615 1312 catalog_manager.cc:4841] TS 3ee520321cbb45a6b757dbedf5839520 (127.28.154.130:39231): tablet 18401d9beffe4bb79279b04a0408f596 (table test-table [id=d177e1d95f8f411096bfd02ea0a230d2]) successfully deleted
I20260811 09:13:09.476672 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 1450
W20260811 09:13:09.508452 1312 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.28.154.129:42473 (error 108)
I20260811 09:13:09.508631 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 1713
I20260811 09:13:09.510377 2228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:09.509869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.129:42473 (local address 127.0.0.1:36988)
0811 09:13:09.510060 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0811 09:13:09.510198 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.129:42473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260811 09:13:09.510758 1312 catalog_manager.cc:4584] TS 9a4a8c7e211e4c50b53c98f28b0da268 (127.28.154.129:42473): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 18401d9beffe4bb79279b04a0408f596: Network error: Client connection negotiation failed: client connection to 127.28.154.129:42473: connect: Connection refused (error 111)
I20260811 09:13:09.517027 2232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:09.509893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60050 (local address 127.28.154.130:39231)
0811 09:13:09.510096 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:09.510106 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:09.510125 (+ 19us) server_negotiation.cc:407] Connection header received
0811 09:13:09.510197 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:09.510203 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:09.510311 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:09.510425 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:09.510925 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:09.511774 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:09.513682 (+ 1908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:09.514146 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:09.514254 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:09.515438 (+ 1184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:13:09.515448 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:13:09.515464 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:13:09.516145 (+ 681us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:13:09.516201 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:13:09.516223 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:13:09.516312 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:09.516551 (+ 239us) server_negotiation.cc:299] Negotiation successful
0811 09:13:09.516646 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260811 09:13:09.517836 1950 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:13:09.564534 2228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:09.564025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.129:42473 (local address 127.0.0.1:37000)
0811 09:13:09.564246 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0811 09:13:09.564356 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.129:42473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260811 09:13:09.565611 2228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:09.565246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.129:42473 (local address 127.0.0.1:37008)
0811 09:13:09.565397 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0811 09:13:09.565470 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.129:42473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260811 09:13:09.626106 2228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:09.625606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.129:42473 (local address 127.0.0.1:37018)
0811 09:13:09.625821 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0811 09:13:09.625915 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.129:42473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260811 09:13:09.627144 2228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:09.626796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.129:42473 (local address 127.0.0.1:37030)
0811 09:13:09.626947 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0811 09:13:09.627017 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.129:42473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260811 09:13:09.630141 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 1713
I20260811 09:13:09.656682 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 1965
I20260811 09:13:09.663472 2230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:09.657471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39028 (local address 127.28.154.131:33875)
0811 09:13:09.657661 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:09.657667 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:09.657726 (+ 59us) server_negotiation.cc:407] Connection header received
0811 09:13:09.657937 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:09.657941 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:09.658028 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:09.658117 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:09.658514 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:09.659222 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:09.660590 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:09.661011 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:09.661116 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:09.662176 (+ 1060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:13:09.662185 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:13:09.662200 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:13:09.662766 (+ 566us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:13:09.662813 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:13:09.662832 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:13:09.662937 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:09.663062 (+ 125us) server_negotiation.cc:299] Negotiation successful
0811 09:13:09.663166 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 09:13:09.664129 2190 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:13:09.708148 2228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:09.707483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.129:42473 (local address 127.0.0.1:37036)
0811 09:13:09.707755 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0811 09:13:09.707931 (+ 176us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.129:42473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
I20260811 09:13:09.709399 2228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:09.708933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.129:42473 (local address 127.0.0.1:37038)
0811 09:13:09.709168 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0811 09:13:09.709258 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.129:42473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110}
I20260811 09:13:09.752832 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 1965
I20260811 09:13:09.775686 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 1276
I20260811 09:13:09.776876 1401 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:13:09.886616 2228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:09.885956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.129:42473 (local address 127.0.0.1:37050)
0811 09:13:09.886271 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0811 09:13:09.886384 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.129:42473: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182}
I20260811 09:13:09.886596 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 1276
2026-08-11T09:13:09Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (3032 ms)
[----------] 6 tests from EncryptionPolicyTest (23349 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786439250192320-29290-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:13:10 dist-test-slave-xt4k krb5kdc[2301](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:13:10 dist-test-slave-xt4k krb5kdc[2301](info): set up 2 sockets
Aug 11 09:13:10 dist-test-slave-xt4k krb5kdc[2301](info): commencing operation
krb5kdc: starting...
W20260811 09:13:11.198501 29290 mini_kdc.cc:121] Time spent starting KDC: real 1.258s user 0.003s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786439250192320-29290-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786439250192320-29290-0/krb5kdc/test-user.keytab.
Aug 11 09:13:11 dist-test-slave-xt4k krb5kdc[2301](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439591, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:13:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:13:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786439250192320-29290-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786439250192320-29290-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 09:13:11.513895 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786439250192320-29290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786439250192320-29290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786439250192320-29290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46015
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786439250192320-29290-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46015
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:13:11.852028 2326 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:13:11.852358 2326 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:13:11.852499 2326 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:13:11.859774 2326 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:13:11.859920 2326 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:13:11.860004 2326 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:13:11.860088 2326 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:13:11.860147 2326 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:13:11.870092 2326 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786439250192320-29290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786439250192320-29290-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46015
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786439250192320-29290-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46015
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786439250192320-29290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:13:11.871426 2326 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:13:11.872821 2326 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:13:11.884586 2333 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:13:11.884652 2332 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:13:11.884624 2336 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:13:11.885241 2326 server_base.cc:1034] running on GCE node
I20260811 09:13:11.886024 2326 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:13:11.887430 2326 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:13:11.888626 2326 hybrid_clock.cc:648] HybridClock initialized: now 1786439591888567 us; error 49 us; skew 500 ppm
I20260811 09:13:11.888988 2326 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:13:11 dist-test-slave-xt4k krb5kdc[2301](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439591, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:13:11.892109 2326 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 09:13:11.893651 2326 webserver.cc:460] Webserver started at http://127.0.0.1:38947/ using document root <none> and password file <none>
I20260811 09:13:11.894317 2326 fs_manager.cc:359] Metadata directory not provided
I20260811 09:13:11.894433 2326 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:13:11.894757 2326 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:13:11.897774 2326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786439250192320-29290-0/minicluster-data/master-0/data/instance:
uuid: "31fd820690ff46299715f845ba96da2d"
format_stamp: "Formatted at 2026-08-11 09:13:11 on dist-test-slave-xt4k"
server_key: "172abef12ace7f949ecb0bae593f6638"
server_key_iv: "18f6477165e9e74d9cec47b56a2d3e2a"
server_key_version: "encryptionkey@0"
I20260811 09:13:11.898542 2326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786439250192320-29290-0/minicluster-data/master-0/wal/instance:
uuid: "31fd820690ff46299715f845ba96da2d"
format_stamp: "Formatted at 2026-08-11 09:13:11 on dist-test-slave-xt4k"
server_key: "172abef12ace7f949ecb0bae593f6638"
server_key_iv: "18f6477165e9e74d9cec47b56a2d3e2a"
server_key_version: "encryptionkey@0"
I20260811 09:13:11.903594 2326 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 09:13:11.907063 2352 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:13:11.908378 2326 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 09:13:11.908546 2326 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786439250192320-29290-0/minicluster-data/master-0/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786439250192320-29290-0/minicluster-data/master-0/wal
uuid: "31fd820690ff46299715f845ba96da2d"
format_stamp: "Formatted at 2026-08-11 09:13:11 on dist-test-slave-xt4k"
server_key: "172abef12ace7f949ecb0bae593f6638"
server_key_iv: "18f6477165e9e74d9cec47b56a2d3e2a"
server_key_version: "encryptionkey@0"
I20260811 09:13:11.908721 2326 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786439250192320-29290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786439250192320-29290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786439250192320-29290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:13:11.921308 2326 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:13:11.925190 2326 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:13:11.925451 2326 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:13:11.942975 2326 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46015
I20260811 09:13:11.943024 2463 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46015 every 8 connection(s)
I20260811 09:13:11.944357 2326 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
I20260811 09:13:11.948086 2467 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:13:11.954510 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 2326
I20260811 09:13:11.954715 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786439250192320-29290-0/minicluster-data/master-0/wal/instance
I20260811 09:13:11.955351 29290 external_mini_cluster.cc:1428] Setting key 3d0094db00e455beb4e1218473154c12
I20260811 09:13:11.959093 2467 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31fd820690ff46299715f845ba96da2d: Bootstrap starting.
I20260811 09:13:11.963073 2467 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 31fd820690ff46299715f845ba96da2d: Neither blocks nor log segments found. Creating new log.
I20260811 09:13:11.964506 2467 log.cc:824] T 00000000000000000000000000000000 P 31fd820690ff46299715f845ba96da2d: Log is configured to *not* fsync() on all Append() calls
I20260811 09:13:11.967902 2467 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31fd820690ff46299715f845ba96da2d: No bootstrap required, opened a new log
Aug 11 09:13:11 dist-test-slave-xt4k krb5kdc[2301](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439591, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 09:13:11.977293 2467 raft_consensus.cc:348] T 00000000000000000000000000000000 P 31fd820690ff46299715f845ba96da2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31fd820690ff46299715f845ba96da2d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46015 } }
I20260811 09:13:11.977720 2467 raft_consensus.cc:374] T 00000000000000000000000000000000 P 31fd820690ff46299715f845ba96da2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:13:11.977892 2467 raft_consensus.cc:729] T 00000000000000000000000000000000 P 31fd820690ff46299715f845ba96da2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31fd820690ff46299715f845ba96da2d, State: Initialized, Role: FOLLOWER
I20260811 09:13:11.978622 2467 consensus_queue.cc:260] T 00000000000000000000000000000000 P 31fd820690ff46299715f845ba96da2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31fd820690ff46299715f845ba96da2d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46015 } }
I20260811 09:13:11.978932 2467 raft_consensus.cc:388] T 00000000000000000000000000000000 P 31fd820690ff46299715f845ba96da2d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:13:11.979079 2467 raft_consensus.cc:482] T 00000000000000000000000000000000 P 31fd820690ff46299715f845ba96da2d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:13:11.979259 2467 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 31fd820690ff46299715f845ba96da2d [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:13:11.980696 2472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:11.957314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33533 (local address 127.0.0.1:46015)
0811 09:13:11.958109 (+ 795us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:11.958124 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:11.958177 (+ 53us) server_negotiation.cc:407] Connection header received
0811 09:13:11.960587 (+ 2410us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:11.960619 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:11.961183 (+ 564us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:11.961647 (+ 464us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:11.963209 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:11.965279 (+ 2070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:11.966803 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:11.967849 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:11.968046 (+ 197us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:11.972054 (+ 4008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:13:11.972066 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:13:11.972091 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:13:11.972144 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:13:11.976281 (+ 4137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:13:11.977098 (+ 817us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:13:11.977106 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:13:11.977114 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:13:11.977331 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:13:11.977787 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:13:11.977792 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:13:11.977795 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:13:11.978517 (+ 722us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:13:11.978841 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:11.979296 (+ 455us) server_negotiation.cc:299] Negotiation successful
0811 09:13:11.979601 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":554,"thread_start_us":328,"threads_started":1}
I20260811 09:13:11.981832 2467 raft_consensus.cc:504] T 00000000000000000000000000000000 P 31fd820690ff46299715f845ba96da2d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31fd820690ff46299715f845ba96da2d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46015 } }
I20260811 09:13:11.982331 2467 leader_election.cc:302] T 00000000000000000000000000000000 P 31fd820690ff46299715f845ba96da2d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 31fd820690ff46299715f845ba96da2d; no voters:
I20260811 09:13:11.983021 2467 leader_election.cc:288] T 00000000000000000000000000000000 P 31fd820690ff46299715f845ba96da2d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:13:11.983124 2475 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 31fd820690ff46299715f845ba96da2d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:13:11.984354 2475 raft_consensus.cc:686] T 00000000000000000000000000000000 P 31fd820690ff46299715f845ba96da2d [term 1 LEADER]: Becoming Leader. State: Replica: 31fd820690ff46299715f845ba96da2d, State: Running, Role: LEADER
I20260811 09:13:11.984853 29290 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 09:13:11.984985 2475 consensus_queue.cc:237] T 00000000000000000000000000000000 P 31fd820690ff46299715f845ba96da2d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31fd820690ff46299715f845ba96da2d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46015 } }
I20260811 09:13:11.985509 2467 sys_catalog.cc:564] T 00000000000000000000000000000000 P 31fd820690ff46299715f845ba96da2d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:13:11.991802 2480 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31fd820690ff46299715f845ba96da2d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "31fd820690ff46299715f845ba96da2d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31fd820690ff46299715f845ba96da2d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46015 } } }
I20260811 09:13:11.992179 2480 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31fd820690ff46299715f845ba96da2d [sys.catalog]: This master's current role is: LEADER
I20260811 09:13:11.993040 2479 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31fd820690ff46299715f845ba96da2d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 31fd820690ff46299715f845ba96da2d. Latest consensus state: current_term: 1 leader_uuid: "31fd820690ff46299715f845ba96da2d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31fd820690ff46299715f845ba96da2d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46015 } } }
I20260811 09:13:11.993427 2479 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31fd820690ff46299715f845ba96da2d [sys.catalog]: This master's current role is: LEADER
I20260811 09:13:11.994071 2499 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:13:12.000178 2499 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:13:12.011473 2472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:11.991711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44637 (local address 127.0.0.1:46015)
0811 09:13:11.992457 (+ 746us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:11.992462 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:11.992496 (+ 34us) server_negotiation.cc:407] Connection header received
0811 09:13:11.992553 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:11.992558 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:11.992667 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:11.992778 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:11.994732 (+ 1954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:11.995909 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:11.997981 (+ 2072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:11.998765 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:11.998897 (+ 132us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:12.002448 (+ 3551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:13:12.002456 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:13:12.002460 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:13:12.002498 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:13:12.006630 (+ 4132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:13:12.007758 (+ 1128us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:13:12.007765 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:13:12.007772 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:13:12.007941 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:13:12.008598 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:13:12.008604 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:13:12.008608 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:13:12.009096 (+ 488us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:13:12.009406 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:12.010779 (+ 1373us) server_negotiation.cc:299] Negotiation successful
0811 09:13:12.011038 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":594}
I20260811 09:13:12.023852 2499 catalog_manager.cc:1347] Generated new cluster ID: 14c9fdae572a4459b75033f8db8153cd
I20260811 09:13:12.023985 2499 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:13:12.038050 2499 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:13:12.039427 2499 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:13:12.047842 2499 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 31fd820690ff46299715f845ba96da2d: Generated new TSK 0
I20260811 09:13:12.048993 2499 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 09:13:12.055656 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 2326
I20260811 09:13:12.057375 2457 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:13:12.148500 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 2326
W20260811 09:13:12.170851 2299 connection.cc:535] client connection to 127.0.0.1:46015 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46015): Connection reset by peer (error 104)
2026-08-11T09:13:12Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (2249 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786439250192320-29290-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:13:12 dist-test-slave-xt4k krb5kdc[2522](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:13:12 dist-test-slave-xt4k krb5kdc[2522](info): set up 2 sockets
Aug 11 09:13:12 dist-test-slave-xt4k krb5kdc[2522](info): commencing operation
krb5kdc: starting...
W20260811 09:13:13.431726 29290 mini_kdc.cc:121] Time spent starting KDC: real 1.242s user 0.004s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786439250192320-29290-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786439250192320-29290-0/krb5kdc/test-user.keytab.
Aug 11 09:13:13 dist-test-slave-xt4k krb5kdc[2522](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439593, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:13:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:13:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786439250192320-29290-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786439250192320-29290-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 09:13:13.747845 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786439250192320-29290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786439250192320-29290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786439250192320-29290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40539
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786439250192320-29290-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40539
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:13:14.091888 2555 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:13:14.092231 2555 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:13:14.092406 2555 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:13:14.099866 2555 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:13:14.100009 2555 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:13:14.100092 2555 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:13:14.100178 2555 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:13:14.100237 2555 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:13:14.110517 2555 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786439250192320-29290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786439250192320-29290-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40539
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786439250192320-29290-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40539
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786439250192320-29290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:13:14.112102 2555 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:13:14.113533 2555 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:13:14.125929 2568 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:13:14.125943 2570 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:13:14.125943 2575 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:13:14.126847 2555 server_base.cc:1034] running on GCE node
I20260811 09:13:14.127674 2555 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:13:14.129249 2555 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:13:14.130471 2555 hybrid_clock.cc:648] HybridClock initialized: now 1786439594130425 us; error 35 us; skew 500 ppm
I20260811 09:13:14.130836 2555 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:13:14 dist-test-slave-xt4k krb5kdc[2522](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439594, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:13:14.139848 2555 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 09:13:14.147313 2555 webserver.cc:460] Webserver started at http://127.0.0.1:38861/ using document root <none> and password file <none>
I20260811 09:13:14.148049 2555 fs_manager.cc:359] Metadata directory not provided
I20260811 09:13:14.148161 2555 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:13:14.148481 2555 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:13:14.151580 2555 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786439250192320-29290-0/minicluster-data/master-0/data/instance:
uuid: "051ab1e6ec994047bbefc024d9727685"
format_stamp: "Formatted at 2026-08-11 09:13:14 on dist-test-slave-xt4k"
server_key: "f62188337fc1d852eecc50c09df71d92"
server_key_iv: "bcbf669a2c3ce04b2b155329f0e1ac4a"
server_key_version: "encryptionkey@0"
I20260811 09:13:14.152320 2555 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786439250192320-29290-0/minicluster-data/master-0/wal/instance:
uuid: "051ab1e6ec994047bbefc024d9727685"
format_stamp: "Formatted at 2026-08-11 09:13:14 on dist-test-slave-xt4k"
server_key: "f62188337fc1d852eecc50c09df71d92"
server_key_iv: "bcbf669a2c3ce04b2b155329f0e1ac4a"
server_key_version: "encryptionkey@0"
I20260811 09:13:14.157792 2555 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260811 09:13:14.161442 2588 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:13:14.162882 2555 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 09:13:14.163060 2555 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786439250192320-29290-0/minicluster-data/master-0/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786439250192320-29290-0/minicluster-data/master-0/wal
uuid: "051ab1e6ec994047bbefc024d9727685"
format_stamp: "Formatted at 2026-08-11 09:13:14 on dist-test-slave-xt4k"
server_key: "f62188337fc1d852eecc50c09df71d92"
server_key_iv: "bcbf669a2c3ce04b2b155329f0e1ac4a"
server_key_version: "encryptionkey@0"
I20260811 09:13:14.163275 2555 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786439250192320-29290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786439250192320-29290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786439250192320-29290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:13:14.179359 2555 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:13:14.183550 2555 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:13:14.183816 2555 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:13:14.201694 2555 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40539
I20260811 09:13:14.201735 2694 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40539 every 8 connection(s)
I20260811 09:13:14.203073 2555 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
I20260811 09:13:14.206902 2698 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:13:14.209110 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 2555
I20260811 09:13:14.209291 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786439250192320-29290-0/minicluster-data/master-0/wal/instance
I20260811 09:13:14.209834 29290 external_mini_cluster.cc:1428] Setting key dc0ba21955ebf278c4e67aeab7dd37b8
I20260811 09:13:14.219193 2698 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 051ab1e6ec994047bbefc024d9727685: Bootstrap starting.
I20260811 09:13:14.222888 2698 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 051ab1e6ec994047bbefc024d9727685: Neither blocks nor log segments found. Creating new log.
I20260811 09:13:14.224042 2698 log.cc:824] T 00000000000000000000000000000000 P 051ab1e6ec994047bbefc024d9727685: Log is configured to *not* fsync() on all Append() calls
I20260811 09:13:14.224229 2702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:14.211873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43889 (local address 127.0.0.1:40539)
0811 09:13:14.213000 (+ 1127us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:14.213013 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:14.213185 (+ 172us) server_negotiation.cc:407] Connection header received
0811 09:13:14.215163 (+ 1978us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:14.215187 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:14.215603 (+ 416us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:14.215946 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:14.216542 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:14.217857 (+ 1315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:14.219576 (+ 1719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:14.220273 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:14.220463 (+ 190us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:14.221787 (+ 1324us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:13:14.221796 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:13:14.221812 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:13:14.222375 (+ 563us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:13:14.222457 (+ 82us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:13:14.222488 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:13:14.222612 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:14.222949 (+ 337us) server_negotiation.cc:299] Negotiation successful
0811 09:13:14.223174 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":854,"thread_start_us":447,"threads_started":1}
I20260811 09:13:14.227242 2698 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 051ab1e6ec994047bbefc024d9727685: No bootstrap required, opened a new log
I20260811 09:13:14.228374 29290 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 09:13:14.236069 2698 raft_consensus.cc:348] T 00000000000000000000000000000000 P 051ab1e6ec994047bbefc024d9727685 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "051ab1e6ec994047bbefc024d9727685" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40539 } }
I20260811 09:13:14.236510 2698 raft_consensus.cc:374] T 00000000000000000000000000000000 P 051ab1e6ec994047bbefc024d9727685 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:13:14.236637 2698 raft_consensus.cc:729] T 00000000000000000000000000000000 P 051ab1e6ec994047bbefc024d9727685 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 051ab1e6ec994047bbefc024d9727685, State: Initialized, Role: FOLLOWER
I20260811 09:13:14.237311 2698 consensus_queue.cc:260] T 00000000000000000000000000000000 P 051ab1e6ec994047bbefc024d9727685 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "051ab1e6ec994047bbefc024d9727685" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40539 } }
I20260811 09:13:14.237571 2698 raft_consensus.cc:388] T 00000000000000000000000000000000 P 051ab1e6ec994047bbefc024d9727685 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:13:14.237711 2698 raft_consensus.cc:482] T 00000000000000000000000000000000 P 051ab1e6ec994047bbefc024d9727685 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:13:14.237867 2698 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 051ab1e6ec994047bbefc024d9727685 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:13:14.239014 2702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:14.232619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55793 (local address 127.0.0.1:40539)
0811 09:13:14.232823 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:14.232829 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:14.233190 (+ 361us) server_negotiation.cc:407] Connection header received
0811 09:13:14.233318 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:14.233327 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:14.233400 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:14.233500 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:14.233944 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:14.235115 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:14.236432 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:14.237062 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:14.237180 (+ 118us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:14.238229 (+ 1049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:13:14.238235 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:13:14.238240 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:13:14.238263 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:13:14.238326 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:13:14.238350 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:13:14.238474 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:14.238617 (+ 143us) server_negotiation.cc:299] Negotiation successful
0811 09:13:14.238676 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 09:13:14.241012 2698 raft_consensus.cc:504] T 00000000000000000000000000000000 P 051ab1e6ec994047bbefc024d9727685 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "051ab1e6ec994047bbefc024d9727685" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40539 } }
I20260811 09:13:14.241508 2698 leader_election.cc:302] T 00000000000000000000000000000000 P 051ab1e6ec994047bbefc024d9727685 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 051ab1e6ec994047bbefc024d9727685; no voters:
I20260811 09:13:14.242069 2698 leader_election.cc:288] T 00000000000000000000000000000000 P 051ab1e6ec994047bbefc024d9727685 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:13:14.242218 2720 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 051ab1e6ec994047bbefc024d9727685 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:13:14.242908 2720 raft_consensus.cc:686] T 00000000000000000000000000000000 P 051ab1e6ec994047bbefc024d9727685 [term 1 LEADER]: Becoming Leader. State: Replica: 051ab1e6ec994047bbefc024d9727685, State: Running, Role: LEADER
I20260811 09:13:14.243510 2720 consensus_queue.cc:237] T 00000000000000000000000000000000 P 051ab1e6ec994047bbefc024d9727685 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "051ab1e6ec994047bbefc024d9727685" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40539 } }
I20260811 09:13:14.244182 2698 sys_catalog.cc:564] T 00000000000000000000000000000000 P 051ab1e6ec994047bbefc024d9727685 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:13:14.246178 2725 sys_catalog.cc:455] T 00000000000000000000000000000000 P 051ab1e6ec994047bbefc024d9727685 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 051ab1e6ec994047bbefc024d9727685. Latest consensus state: current_term: 1 leader_uuid: "051ab1e6ec994047bbefc024d9727685" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "051ab1e6ec994047bbefc024d9727685" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40539 } } }
I20260811 09:13:14.246773 2724 sys_catalog.cc:455] T 00000000000000000000000000000000 P 051ab1e6ec994047bbefc024d9727685 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "051ab1e6ec994047bbefc024d9727685" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "051ab1e6ec994047bbefc024d9727685" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40539 } } }
I20260811 09:13:14.247087 2725 sys_catalog.cc:458] T 00000000000000000000000000000000 P 051ab1e6ec994047bbefc024d9727685 [sys.catalog]: This master's current role is: LEADER
I20260811 09:13:14.247192 2724 sys_catalog.cc:458] T 00000000000000000000000000000000 P 051ab1e6ec994047bbefc024d9727685 [sys.catalog]: This master's current role is: LEADER
I20260811 09:13:14.248494 2732 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:13:14.254462 2732 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:13:14.263612 2732 catalog_manager.cc:1347] Generated new cluster ID: 7c69bc8bfcd54831b3d29ee30e71b9aa
I20260811 09:13:14.263823 2732 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:13:14.277529 2732 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:13:14.279364 2732 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:13:14.290226 2732 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 051ab1e6ec994047bbefc024d9727685: Generated new TSK 0
I20260811 09:13:14.291113 2732 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 09:13:14.299343 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 2555
W20260811 09:13:14.300549 2686 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:43889
W20260811 09:13:14.301416 29290 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 2555: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 09:13:14.301607 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 2555
W20260811 09:13:14.333873 2518 connection.cc:535] client connection to 127.0.0.1:40539 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40539): Connection reset by peer (error 104)
2026-08-11T09:13:14Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (2163 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786439250192320-29290-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:13:14 dist-test-slave-xt4k krb5kdc[2758](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:13:14 dist-test-slave-xt4k krb5kdc[2758](info): set up 2 sockets
Aug 11 09:13:14 dist-test-slave-xt4k krb5kdc[2758](info): commencing operation
krb5kdc: starting...
W20260811 09:13:15.611792 29290 mini_kdc.cc:121] Time spent starting KDC: real 1.259s user 0.000s sys 0.048s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786439250192320-29290-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786439250192320-29290-0/krb5kdc/test-user.keytab.
Aug 11 09:13:15 dist-test-slave-xt4k krb5kdc[2758](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439595, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:13:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:13:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786439250192320-29290-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786439250192320-29290-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 09:13:15.935720 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786439250192320-29290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786439250192320-29290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786439250192320-29290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37135
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786439250192320-29290-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37135
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:13:16.280687 2775 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:13:16.280979 2775 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:13:16.281116 2775 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:13:16.288300 2775 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:13:16.288450 2775 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:13:16.288532 2775 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:13:16.288618 2775 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:13:16.288669 2775 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:13:16.299835 2775 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786439250192320-29290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786439250192320-29290-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37135
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786439250192320-29290-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37135
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786439250192320-29290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:13:16.301158 2775 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:13:16.302496 2775 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:13:16.314059 2799 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:13:16.314121 2796 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:13:16.314736 2775 server_base.cc:1034] running on GCE node
W20260811 09:13:16.314764 2795 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:13:16.315574 2775 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:13:16.317044 2775 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:13:16.318269 2775 hybrid_clock.cc:648] HybridClock initialized: now 1786439596318222 us; error 39 us; skew 500 ppm
I20260811 09:13:16.318625 2775 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:13:16 dist-test-slave-xt4k krb5kdc[2758](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439596, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:13:16.321789 2775 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 09:13:16.323297 2775 webserver.cc:460] Webserver started at http://127.0.0.1:41005/ using document root <none> and password file <none>
I20260811 09:13:16.323963 2775 fs_manager.cc:359] Metadata directory not provided
I20260811 09:13:16.324082 2775 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:13:16.324424 2775 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:13:16.327397 2775 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786439250192320-29290-0/minicluster-data/master-0/data/instance:
uuid: "f8aab6c4550e47f99575bc67d202d032"
format_stamp: "Formatted at 2026-08-11 09:13:16 on dist-test-slave-xt4k"
server_key: "e24cb4bff39f1d0d34514dec25a4ae26"
server_key_iv: "a2b8b7af6b1ca55287aa1803eff46a43"
server_key_version: "encryptionkey@0"
I20260811 09:13:16.328159 2775 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786439250192320-29290-0/minicluster-data/master-0/wal/instance:
uuid: "f8aab6c4550e47f99575bc67d202d032"
format_stamp: "Formatted at 2026-08-11 09:13:16 on dist-test-slave-xt4k"
server_key: "e24cb4bff39f1d0d34514dec25a4ae26"
server_key_iv: "a2b8b7af6b1ca55287aa1803eff46a43"
server_key_version: "encryptionkey@0"
I20260811 09:13:16.333417 2775 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260811 09:13:16.337005 2819 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:13:16.338531 2775 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 09:13:16.338714 2775 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786439250192320-29290-0/minicluster-data/master-0/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786439250192320-29290-0/minicluster-data/master-0/wal
uuid: "f8aab6c4550e47f99575bc67d202d032"
format_stamp: "Formatted at 2026-08-11 09:13:16 on dist-test-slave-xt4k"
server_key: "e24cb4bff39f1d0d34514dec25a4ae26"
server_key_iv: "a2b8b7af6b1ca55287aa1803eff46a43"
server_key_version: "encryptionkey@0"
I20260811 09:13:16.338902 2775 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786439250192320-29290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786439250192320-29290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786439250192320-29290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:13:16.344754 2775 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:13:16.348897 2775 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:13:16.349179 2775 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:13:16.366873 2775 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37135
I20260811 09:13:16.366945 2914 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37135 every 8 connection(s)
I20260811 09:13:16.368247 2775 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
I20260811 09:13:16.370268 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 2775
I20260811 09:13:16.370433 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786439250192320-29290-0/minicluster-data/master-0/wal/instance
I20260811 09:13:16.370831 29290 external_mini_cluster.cc:1428] Setting key c8669e95d9b537271e7b67c60f8e840c
I20260811 09:13:16.372182 2916 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:13:16.378341 2920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:16.372600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49339 (local address 127.0.0.1:37135)
0811 09:13:16.373379 (+ 779us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:16.373390 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:16.373431 (+ 41us) server_negotiation.cc:407] Connection header received
0811 09:13:16.375012 (+ 1581us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:16.375033 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:16.375363 (+ 330us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:16.375723 (+ 360us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:16.376771 (+ 1048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:13:16.376777 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:13:16.376790 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:13:16.376813 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:13:16.376891 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:13:16.376911 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:13:16.377137 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:16.377275 (+ 138us) server_negotiation.cc:299] Negotiation successful
0811 09:13:16.377446 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":564,"thread_start_us":338,"threads_started":1}
I20260811 09:13:16.381870 29290 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 09:13:16.384158 2916 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8aab6c4550e47f99575bc67d202d032: Bootstrap starting.
I20260811 09:13:16.387694 2916 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f8aab6c4550e47f99575bc67d202d032: Neither blocks nor log segments found. Creating new log.
I20260811 09:13:16.388485 2920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:16.385918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55333 (local address 127.0.0.1:37135)
0811 09:13:16.386098 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:16.386105 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:16.386522 (+ 417us) server_negotiation.cc:407] Connection header received
0811 09:13:16.386659 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:16.386664 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:16.386734 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:16.386851 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:16.387762 (+ 911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:13:16.387768 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:13:16.387772 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:13:16.387793 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:13:16.387843 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:13:16.387864 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:13:16.387987 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:16.388099 (+ 112us) server_negotiation.cc:299] Negotiation successful
0811 09:13:16.388168 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 09:13:16.389077 2916 log.cc:824] T 00000000000000000000000000000000 P f8aab6c4550e47f99575bc67d202d032: Log is configured to *not* fsync() on all Append() calls
I20260811 09:13:16.391743 2916 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8aab6c4550e47f99575bc67d202d032: No bootstrap required, opened a new log
I20260811 09:13:16.399080 2916 raft_consensus.cc:348] T 00000000000000000000000000000000 P f8aab6c4550e47f99575bc67d202d032 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8aab6c4550e47f99575bc67d202d032" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37135 } }
I20260811 09:13:16.399467 2916 raft_consensus.cc:374] T 00000000000000000000000000000000 P f8aab6c4550e47f99575bc67d202d032 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:13:16.399564 2916 raft_consensus.cc:729] T 00000000000000000000000000000000 P f8aab6c4550e47f99575bc67d202d032 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8aab6c4550e47f99575bc67d202d032, State: Initialized, Role: FOLLOWER
I20260811 09:13:16.400133 2916 consensus_queue.cc:260] T 00000000000000000000000000000000 P f8aab6c4550e47f99575bc67d202d032 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8aab6c4550e47f99575bc67d202d032" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37135 } }
I20260811 09:13:16.400373 2916 raft_consensus.cc:388] T 00000000000000000000000000000000 P f8aab6c4550e47f99575bc67d202d032 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:13:16.400516 2916 raft_consensus.cc:482] T 00000000000000000000000000000000 P f8aab6c4550e47f99575bc67d202d032 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:13:16.400664 2916 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f8aab6c4550e47f99575bc67d202d032 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:13:16.402977 2916 raft_consensus.cc:504] T 00000000000000000000000000000000 P f8aab6c4550e47f99575bc67d202d032 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8aab6c4550e47f99575bc67d202d032" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37135 } }
I20260811 09:13:16.403527 2916 leader_election.cc:302] T 00000000000000000000000000000000 P f8aab6c4550e47f99575bc67d202d032 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f8aab6c4550e47f99575bc67d202d032; no voters:
I20260811 09:13:16.404220 2916 leader_election.cc:288] T 00000000000000000000000000000000 P f8aab6c4550e47f99575bc67d202d032 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:13:16.404383 2932 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f8aab6c4550e47f99575bc67d202d032 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:13:16.405009 2932 raft_consensus.cc:686] T 00000000000000000000000000000000 P f8aab6c4550e47f99575bc67d202d032 [term 1 LEADER]: Becoming Leader. State: Replica: f8aab6c4550e47f99575bc67d202d032, State: Running, Role: LEADER
I20260811 09:13:16.405649 2932 consensus_queue.cc:237] T 00000000000000000000000000000000 P f8aab6c4550e47f99575bc67d202d032 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8aab6c4550e47f99575bc67d202d032" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37135 } }
I20260811 09:13:16.406630 2916 sys_catalog.cc:564] T 00000000000000000000000000000000 P f8aab6c4550e47f99575bc67d202d032 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:13:16.408802 2935 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8aab6c4550e47f99575bc67d202d032 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f8aab6c4550e47f99575bc67d202d032" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8aab6c4550e47f99575bc67d202d032" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37135 } } }
I20260811 09:13:16.408830 2937 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8aab6c4550e47f99575bc67d202d032 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f8aab6c4550e47f99575bc67d202d032. Latest consensus state: current_term: 1 leader_uuid: "f8aab6c4550e47f99575bc67d202d032" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f8aab6c4550e47f99575bc67d202d032" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37135 } } }
I20260811 09:13:16.409350 2935 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8aab6c4550e47f99575bc67d202d032 [sys.catalog]: This master's current role is: LEADER
I20260811 09:13:16.409358 2937 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8aab6c4550e47f99575bc67d202d032 [sys.catalog]: This master's current role is: LEADER
I20260811 09:13:16.411031 2946 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:13:16.415736 2946 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:13:16.424044 2946 catalog_manager.cc:1347] Generated new cluster ID: 0bba7834d4da4751bfd368f55a776f3c
I20260811 09:13:16.424250 2946 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:13:16.438807 2946 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:13:16.439380 2946 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:13:16.447201 2946 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f8aab6c4550e47f99575bc67d202d032: Generated new TSK 0
I20260811 09:13:16.448263 2946 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 09:13:16.457585 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 2775
W20260811 09:13:16.458977 2907 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:49339
W20260811 09:13:16.459668 29290 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 2775: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 09:13:16.459810 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 2775
2026-08-11T09:13:16Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (2155 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786439250192320-29290-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:13:16 dist-test-slave-xt4k krb5kdc[2970](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:13:16 dist-test-slave-xt4k krb5kdc[2970](info): set up 2 sockets
Aug 11 09:13:16 dist-test-slave-xt4k krb5kdc[2970](info): commencing operation
krb5kdc: starting...
W20260811 09:13:17.778836 29290 mini_kdc.cc:121] Time spent starting KDC: real 1.270s user 0.000s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786439250192320-29290-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786439250192320-29290-0/krb5kdc/test-user.keytab.
Aug 11 09:13:17 dist-test-slave-xt4k krb5kdc[2970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439597, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:13:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:13:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786439250192320-29290-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786439250192320-29290-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 09:13:18.108012 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786439250192320-29290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786439250192320-29290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786439250192320-29290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44025
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786439250192320-29290-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44025
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:13:18.477648 2992 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:13:18.477993 2992 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:13:18.478251 2992 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:13:18.486287 2992 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:13:18.486436 2992 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:13:18.486483 2992 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:13:18.486543 2992 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:13:18.486590 2992 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:13:18.497432 2992 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786439250192320-29290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786439250192320-29290-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44025
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786439250192320-29290-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44025
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786439250192320-29290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:13:18.499176 2992 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:13:18.500587 2992 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:13:18.513536 3006 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:13:18.513443 3011 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:13:18.513499 3008 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:13:18.514395 2992 server_base.cc:1034] running on GCE node
I20260811 09:13:18.515240 2992 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:13:18.516764 2992 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:13:18.517992 2992 hybrid_clock.cc:648] HybridClock initialized: now 1786439598517931 us; error 50 us; skew 500 ppm
I20260811 09:13:18.518385 2992 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:13:18 dist-test-slave-xt4k krb5kdc[2970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439598, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:13:18.521926 2992 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 09:13:18.523773 2992 webserver.cc:460] Webserver started at http://127.0.0.1:38261/ using document root <none> and password file <none>
I20260811 09:13:18.524957 2992 fs_manager.cc:359] Metadata directory not provided
I20260811 09:13:18.525094 2992 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:13:18.525393 2992 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:13:18.528597 2992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786439250192320-29290-0/minicluster-data/master-0/data/instance:
uuid: "725ad6edec884e858d72266e56e7a802"
format_stamp: "Formatted at 2026-08-11 09:13:18 on dist-test-slave-xt4k"
server_key: "b3e07e0a53b54a300494950961859ea9"
server_key_iv: "d75e3d3710aff6dfa61c427b0206f762"
server_key_version: "encryptionkey@0"
I20260811 09:13:18.529395 2992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786439250192320-29290-0/minicluster-data/master-0/wal/instance:
uuid: "725ad6edec884e858d72266e56e7a802"
format_stamp: "Formatted at 2026-08-11 09:13:18 on dist-test-slave-xt4k"
server_key: "b3e07e0a53b54a300494950961859ea9"
server_key_iv: "d75e3d3710aff6dfa61c427b0206f762"
server_key_version: "encryptionkey@0"
I20260811 09:13:18.534940 2992 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 09:13:18.538771 3028 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:13:18.540215 2992 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 09:13:18.540438 2992 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786439250192320-29290-0/minicluster-data/master-0/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786439250192320-29290-0/minicluster-data/master-0/wal
uuid: "725ad6edec884e858d72266e56e7a802"
format_stamp: "Formatted at 2026-08-11 09:13:18 on dist-test-slave-xt4k"
server_key: "b3e07e0a53b54a300494950961859ea9"
server_key_iv: "d75e3d3710aff6dfa61c427b0206f762"
server_key_version: "encryptionkey@0"
I20260811 09:13:18.540650 2992 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786439250192320-29290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786439250192320-29290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786439250192320-29290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:13:18.562664 2992 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:13:18.569020 2992 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:13:18.569337 2992 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:13:18.589848 2992 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44025
I20260811 09:13:18.589882 3126 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44025 every 8 connection(s)
I20260811 09:13:18.591357 2992 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
I20260811 09:13:18.592849 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 2992
I20260811 09:13:18.593044 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786439250192320-29290-0/minicluster-data/master-0/wal/instance
I20260811 09:13:18.593608 29290 external_mini_cluster.cc:1428] Setting key 99ca5420799f601a2ebebf234bafb483
I20260811 09:13:18.596050 3128 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 09:13:18 dist-test-slave-xt4k krb5kdc[2970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439597, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 09:13:18.610186 3128 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 725ad6edec884e858d72266e56e7a802: Bootstrap starting.
I20260811 09:13:18.614734 3128 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 725ad6edec884e858d72266e56e7a802: Neither blocks nor log segments found. Creating new log.
I20260811 09:13:18.615975 3128 log.cc:824] T 00000000000000000000000000000000 P 725ad6edec884e858d72266e56e7a802: Log is configured to *not* fsync() on all Append() calls
I20260811 09:13:18.619663 3128 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 725ad6edec884e858d72266e56e7a802: No bootstrap required, opened a new log
I20260811 09:13:18.620213 3132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:18.595735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56691 (local address 127.0.0.1:44025)
0811 09:13:18.596747 (+ 1012us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:18.596763 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:18.597118 (+ 355us) server_negotiation.cc:407] Connection header received
0811 09:13:18.599332 (+ 2214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:18.599362 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:18.599919 (+ 557us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:18.600423 (+ 504us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:18.601714 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:18.603850 (+ 2136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:18.605301 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:18.606365 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:18.606581 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:18.611158 (+ 4577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:13:18.611169 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:13:18.611189 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:13:18.611238 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:13:18.615226 (+ 3988us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:13:18.616118 (+ 892us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:13:18.616127 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:13:18.616135 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:13:18.616387 (+ 252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:13:18.616889 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:13:18.616894 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:13:18.616896 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:13:18.617665 (+ 769us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:13:18.617970 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:18.618414 (+ 444us) server_negotiation.cc:299] Negotiation successful
0811 09:13:18.618877 (+ 463us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":744,"thread_start_us":455,"threads_started":1}
I20260811 09:13:18.624095 29290 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 09:13:18.629920 3128 raft_consensus.cc:348] T 00000000000000000000000000000000 P 725ad6edec884e858d72266e56e7a802 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "725ad6edec884e858d72266e56e7a802" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44025 } }
I20260811 09:13:18.630439 3128 raft_consensus.cc:374] T 00000000000000000000000000000000 P 725ad6edec884e858d72266e56e7a802 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:13:18.630573 3128 raft_consensus.cc:729] T 00000000000000000000000000000000 P 725ad6edec884e858d72266e56e7a802 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 725ad6edec884e858d72266e56e7a802, State: Initialized, Role: FOLLOWER
I20260811 09:13:18.631266 3128 consensus_queue.cc:260] T 00000000000000000000000000000000 P 725ad6edec884e858d72266e56e7a802 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "725ad6edec884e858d72266e56e7a802" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44025 } }
I20260811 09:13:18.631580 3128 raft_consensus.cc:388] T 00000000000000000000000000000000 P 725ad6edec884e858d72266e56e7a802 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:13:18.631723 3128 raft_consensus.cc:482] T 00000000000000000000000000000000 P 725ad6edec884e858d72266e56e7a802 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:13:18.631919 3128 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 725ad6edec884e858d72266e56e7a802 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:13:18.634959 3128 raft_consensus.cc:504] T 00000000000000000000000000000000 P 725ad6edec884e858d72266e56e7a802 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "725ad6edec884e858d72266e56e7a802" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44025 } }
I20260811 09:13:18.635579 3128 leader_election.cc:302] T 00000000000000000000000000000000 P 725ad6edec884e858d72266e56e7a802 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 725ad6edec884e858d72266e56e7a802; no voters:
I20260811 09:13:18.636240 3128 leader_election.cc:288] T 00000000000000000000000000000000 P 725ad6edec884e858d72266e56e7a802 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:13:18.636356 3145 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 725ad6edec884e858d72266e56e7a802 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:13:18.638970 3145 raft_consensus.cc:686] T 00000000000000000000000000000000 P 725ad6edec884e858d72266e56e7a802 [term 1 LEADER]: Becoming Leader. State: Replica: 725ad6edec884e858d72266e56e7a802, State: Running, Role: LEADER
I20260811 09:13:18.639169 3128 sys_catalog.cc:564] T 00000000000000000000000000000000 P 725ad6edec884e858d72266e56e7a802 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:13:18.639595 3145 consensus_queue.cc:237] T 00000000000000000000000000000000 P 725ad6edec884e858d72266e56e7a802 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "725ad6edec884e858d72266e56e7a802" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44025 } }
I20260811 09:13:18.645058 3132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:18.629805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56755 (local address 127.0.0.1:44025)
0811 09:13:18.630014 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:18.630021 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:18.630548 (+ 527us) server_negotiation.cc:407] Connection header received
0811 09:13:18.630750 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:18.630755 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:18.630823 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:18.630953 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:18.632958 (+ 2005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:18.633766 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:18.635432 (+ 1666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:18.635990 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:18.636108 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:18.638793 (+ 2685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:13:18.638800 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:13:18.638808 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:13:18.638851 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:13:18.642150 (+ 3299us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:13:18.642819 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:13:18.642825 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:13:18.642829 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:13:18.643021 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:13:18.643379 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:13:18.643384 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:13:18.643388 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:13:18.643674 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:13:18.643956 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:18.644254 (+ 298us) server_negotiation.cc:299] Negotiation successful
0811 09:13:18.644503 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 09:13:18.649978 3153 sys_catalog.cc:455] T 00000000000000000000000000000000 P 725ad6edec884e858d72266e56e7a802 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 725ad6edec884e858d72266e56e7a802. Latest consensus state: current_term: 1 leader_uuid: "725ad6edec884e858d72266e56e7a802" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "725ad6edec884e858d72266e56e7a802" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44025 } } }
I20260811 09:13:18.650426 3153 sys_catalog.cc:458] T 00000000000000000000000000000000 P 725ad6edec884e858d72266e56e7a802 [sys.catalog]: This master's current role is: LEADER
I20260811 09:13:18.649953 3150 sys_catalog.cc:455] T 00000000000000000000000000000000 P 725ad6edec884e858d72266e56e7a802 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "725ad6edec884e858d72266e56e7a802" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "725ad6edec884e858d72266e56e7a802" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44025 } } }
I20260811 09:13:18.650916 3150 sys_catalog.cc:458] T 00000000000000000000000000000000 P 725ad6edec884e858d72266e56e7a802 [sys.catalog]: This master's current role is: LEADER
I20260811 09:13:18.657982 3170 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:13:18.663271 3170 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:13:18.672410 3170 catalog_manager.cc:1347] Generated new cluster ID: 3c3e7979a173486898496ec074b27695
I20260811 09:13:18.672545 3170 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:13:18.683892 3170 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:13:18.685911 3170 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:13:18.693298 3170 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 725ad6edec884e858d72266e56e7a802: Generated new TSK 0
I20260811 09:13:18.694185 3170 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 09:13:18.703868 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 2992
I20260811 09:13:18.705369 3119 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:13:18.795262 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 2992
2026-08-11T09:13:18Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (2327 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786439250192320-29290-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:13:18 dist-test-slave-xt4k krb5kdc[3189](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:13:18 dist-test-slave-xt4k krb5kdc[3189](info): set up 2 sockets
Aug 11 09:13:18 dist-test-slave-xt4k krb5kdc[3189](info): commencing operation
krb5kdc: starting...
W20260811 09:13:20.083768 29290 mini_kdc.cc:121] Time spent starting KDC: real 1.248s user 0.005s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786439250192320-29290-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786439250192320-29290-0/krb5kdc/test-user.keytab.
Aug 11 09:13:20 dist-test-slave-xt4k krb5kdc[3189](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439600, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:13:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:13:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786439250192320-29290-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786439250192320-29290-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 09:13:20.407475 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786439250192320-29290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786439250192320-29290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786439250192320-29290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42279
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786439250192320-29290-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46275
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42279
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:13:20.756170 3216 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:13:20.756515 3216 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:13:20.756660 3216 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:13:20.763952 3216 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:13:20.764075 3216 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:13:20.764151 3216 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:13:20.764235 3216 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:13:20.764297 3216 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:13:20.774538 3216 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46275
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786439250192320-29290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786439250192320-29290-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42279
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786439250192320-29290-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42279
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786439250192320-29290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:13:20.775872 3216 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:13:20.777235 3216 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:13:20.789224 3232 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:13:20.789551 3241 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:13:20.789589 3229 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:13:20.790421 3216 server_base.cc:1034] running on GCE node
I20260811 09:13:20.791255 3216 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:13:20.792732 3216 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:13:20.794003 3216 hybrid_clock.cc:648] HybridClock initialized: now 1786439600793947 us; error 43 us; skew 500 ppm
I20260811 09:13:20.794363 3216 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:13:20 dist-test-slave-xt4k krb5kdc[3189](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439600, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:13:20.802062 3216 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 09:13:20.803694 3216 webserver.cc:460] Webserver started at http://127.0.0.1:35319/ using document root <none> and password file <none>
I20260811 09:13:20.804471 3216 fs_manager.cc:359] Metadata directory not provided
I20260811 09:13:20.804606 3216 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:13:20.804926 3216 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:13:20.808125 3216 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786439250192320-29290-0/minicluster-data/master-0/data/instance:
uuid: "2d75c7b4945f4797ab732736a0896344"
format_stamp: "Formatted at 2026-08-11 09:13:20 on dist-test-slave-xt4k"
server_key: "fa994405217b2144acbd91582384a920"
server_key_iv: "ae4f853721bdd7837f324f7e66f4e6e6"
server_key_version: "encryptionkey@0"
I20260811 09:13:20.809023 3216 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786439250192320-29290-0/minicluster-data/master-0/wal/instance:
uuid: "2d75c7b4945f4797ab732736a0896344"
format_stamp: "Formatted at 2026-08-11 09:13:20 on dist-test-slave-xt4k"
server_key: "fa994405217b2144acbd91582384a920"
server_key_iv: "ae4f853721bdd7837f324f7e66f4e6e6"
server_key_version: "encryptionkey@0"
I20260811 09:13:20.814425 3216 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 09:13:20.818194 3261 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:13:20.819620 3216 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 09:13:20.819811 3216 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786439250192320-29290-0/minicluster-data/master-0/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786439250192320-29290-0/minicluster-data/master-0/wal
uuid: "2d75c7b4945f4797ab732736a0896344"
format_stamp: "Formatted at 2026-08-11 09:13:20 on dist-test-slave-xt4k"
server_key: "fa994405217b2144acbd91582384a920"
server_key_iv: "ae4f853721bdd7837f324f7e66f4e6e6"
server_key_version: "encryptionkey@0"
I20260811 09:13:20.820006 3216 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786439250192320-29290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786439250192320-29290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786439250192320-29290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:13:20.840535 3216 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:13:20.844821 3216 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:13:20.845098 3216 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:13:20.862213 3216 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42279
I20260811 09:13:20.862243 3349 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42279 every 8 connection(s)
I20260811 09:13:20.863785 3216 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
I20260811 09:13:20.867583 3350 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:13:20.869724 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 3216
I20260811 09:13:20.869902 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786439250192320-29290-0/minicluster-data/master-0/wal/instance
I20260811 09:13:20.870424 29290 external_mini_cluster.cc:1428] Setting key d0b36e2f0b510b6e8697bb7209ae830a
I20260811 09:13:20.880245 3350 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d75c7b4945f4797ab732736a0896344: Bootstrap starting.
I20260811 09:13:20.883958 3350 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2d75c7b4945f4797ab732736a0896344: Neither blocks nor log segments found. Creating new log.
I20260811 09:13:20.884449 3360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:20.872289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40511 (local address 127.0.0.1:42279)
0811 09:13:20.873155 (+ 866us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:20.873168 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:20.873216 (+ 48us) server_negotiation.cc:407] Connection header received
0811 09:13:20.874950 (+ 1734us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:20.874983 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:20.875402 (+ 419us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:20.875727 (+ 325us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:20.876405 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:20.877865 (+ 1460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:20.879675 (+ 1810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:20.880532 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:20.880673 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:20.881980 (+ 1307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:13:20.881990 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:13:20.882008 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:13:20.882613 (+ 605us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:13:20.882696 (+ 83us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:13:20.882727 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:13:20.882812 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:20.883058 (+ 246us) server_negotiation.cc:299] Negotiation successful
0811 09:13:20.883308 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":628,"thread_start_us":360,"threads_started":1}
I20260811 09:13:20.885197 3350 log.cc:824] T 00000000000000000000000000000000 P 2d75c7b4945f4797ab732736a0896344: Log is configured to *not* fsync() on all Append() calls
I20260811 09:13:20.888028 3350 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d75c7b4945f4797ab732736a0896344: No bootstrap required, opened a new log
I20260811 09:13:20.888041 29290 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 09:13:20.896833 3350 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2d75c7b4945f4797ab732736a0896344 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d75c7b4945f4797ab732736a0896344" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42279 } }
I20260811 09:13:20.897354 3350 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2d75c7b4945f4797ab732736a0896344 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:13:20.897498 3350 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2d75c7b4945f4797ab732736a0896344 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d75c7b4945f4797ab732736a0896344, State: Initialized, Role: FOLLOWER
I20260811 09:13:20.898185 3350 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2d75c7b4945f4797ab732736a0896344 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d75c7b4945f4797ab732736a0896344" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42279 } }
I20260811 09:13:20.898507 3350 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2d75c7b4945f4797ab732736a0896344 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:13:20.898658 3350 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2d75c7b4945f4797ab732736a0896344 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:13:20.898844 3350 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2d75c7b4945f4797ab732736a0896344 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:13:20.901616 3350 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2d75c7b4945f4797ab732736a0896344 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d75c7b4945f4797ab732736a0896344" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42279 } }
I20260811 09:13:20.902195 3350 leader_election.cc:302] T 00000000000000000000000000000000 P 2d75c7b4945f4797ab732736a0896344 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2d75c7b4945f4797ab732736a0896344; no voters:
I20260811 09:13:20.903064 3360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:20.893196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37283 (local address 127.0.0.1:42279)
0811 09:13:20.893394 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:20.893400 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:20.896958 (+ 3558us) server_negotiation.cc:407] Connection header received
0811 09:13:20.897115 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:20.897121 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:20.897212 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:20.897320 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:20.897975 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:20.899058 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:20.900371 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:20.901062 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:20.901178 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:20.902294 (+ 1116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:13:20.902301 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:13:20.902306 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:13:20.902329 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:13:20.902388 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:13:20.902406 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:13:20.902502 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:20.902619 (+ 117us) server_negotiation.cc:299] Negotiation successful
0811 09:13:20.902720 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 09:13:20.903178 3350 leader_election.cc:288] T 00000000000000000000000000000000 P 2d75c7b4945f4797ab732736a0896344 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:13:20.903277 3372 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2d75c7b4945f4797ab732736a0896344 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:13:20.904968 3372 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2d75c7b4945f4797ab732736a0896344 [term 1 LEADER]: Becoming Leader. State: Replica: 2d75c7b4945f4797ab732736a0896344, State: Running, Role: LEADER
I20260811 09:13:20.905537 3350 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2d75c7b4945f4797ab732736a0896344 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:13:20.905544 3372 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2d75c7b4945f4797ab732736a0896344 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d75c7b4945f4797ab732736a0896344" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42279 } }
I20260811 09:13:20.908051 3375 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d75c7b4945f4797ab732736a0896344 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2d75c7b4945f4797ab732736a0896344. Latest consensus state: current_term: 1 leader_uuid: "2d75c7b4945f4797ab732736a0896344" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d75c7b4945f4797ab732736a0896344" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42279 } } }
I20260811 09:13:20.908159 3374 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d75c7b4945f4797ab732736a0896344 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2d75c7b4945f4797ab732736a0896344" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d75c7b4945f4797ab732736a0896344" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42279 } } }
I20260811 09:13:20.908448 3375 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d75c7b4945f4797ab732736a0896344 [sys.catalog]: This master's current role is: LEADER
I20260811 09:13:20.908537 3374 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d75c7b4945f4797ab732736a0896344 [sys.catalog]: This master's current role is: LEADER
I20260811 09:13:20.911974 3387 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:13:20.917951 3387 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:13:20.928217 3387 catalog_manager.cc:1347] Generated new cluster ID: a1e8eb217eac40d58d9c17d6ca6e2fa2
I20260811 09:13:20.928373 3387 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:13:20.942858 3387 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:13:20.944885 3387 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:13:20.959263 3387 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2d75c7b4945f4797ab732736a0896344: Generated new TSK 0
I20260811 09:13:20.960523 3387 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 09:13:20.973263 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 3216
W20260811 09:13:20.974630 3335 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:40511
W20260811 09:13:20.975378 29290 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 3216: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 09:13:20.975512 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 3216
2026-08-11T09:13:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (2183 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786439250192320-29290-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:13:21 dist-test-slave-xt4k krb5kdc[3410](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:13:21 dist-test-slave-xt4k krb5kdc[3410](info): set up 2 sockets
Aug 11 09:13:21 dist-test-slave-xt4k krb5kdc[3410](info): commencing operation
krb5kdc: starting...
W20260811 09:13:22.289316 29290 mini_kdc.cc:121] Time spent starting KDC: real 1.269s user 0.000s sys 0.050s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786439250192320-29290-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786439250192320-29290-0/krb5kdc/test-user.keytab.
Aug 11 09:13:22 dist-test-slave-xt4k krb5kdc[3410](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439602, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:13:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:13:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786439250192320-29290-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786439250192320-29290-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 09:13:22.621708 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786439250192320-29290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786439250192320-29290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786439250192320-29290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36643
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786439250192320-29290-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40655
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36643
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:13:22.973961 3435 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:13:22.974277 3435 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:13:22.974427 3435 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:13:22.981789 3435 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:13:22.981930 3435 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:13:22.982014 3435 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:13:22.982110 3435 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:13:22.982170 3435 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:13:22.992753 3435 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40655
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786439250192320-29290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786439250192320-29290-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36643
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786439250192320-29290-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36643
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786439250192320-29290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:13:22.994230 3435 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:13:22.995671 3435 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:13:23.007977 3447 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:13:23.007946 3451 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:13:23.008037 3448 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:13:23.009296 3435 server_base.cc:1034] running on GCE node
I20260811 09:13:23.010174 3435 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:13:23.011763 3435 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:13:23.012996 3435 hybrid_clock.cc:648] HybridClock initialized: now 1786439603012945 us; error 43 us; skew 500 ppm
I20260811 09:13:23.013370 3435 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:13:23 dist-test-slave-xt4k krb5kdc[3410](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439603, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:13:23.016712 3435 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 09:13:23.018311 3435 webserver.cc:460] Webserver started at http://127.0.0.1:39977/ using document root <none> and password file <none>
I20260811 09:13:23.019029 3435 fs_manager.cc:359] Metadata directory not provided
I20260811 09:13:23.019157 3435 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:13:23.019490 3435 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:13:23.022662 3435 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786439250192320-29290-0/minicluster-data/master-0/data/instance:
uuid: "31b1cd89c5b144ac82b3af8ae812f770"
format_stamp: "Formatted at 2026-08-11 09:13:23 on dist-test-slave-xt4k"
server_key: "49e8198f3b2593545e55c0cb9f0e8500"
server_key_iv: "1d73d7bad12ba8f971ca8a095583dbfd"
server_key_version: "encryptionkey@0"
I20260811 09:13:23.023470 3435 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786439250192320-29290-0/minicluster-data/master-0/wal/instance:
uuid: "31b1cd89c5b144ac82b3af8ae812f770"
format_stamp: "Formatted at 2026-08-11 09:13:23 on dist-test-slave-xt4k"
server_key: "49e8198f3b2593545e55c0cb9f0e8500"
server_key_iv: "1d73d7bad12ba8f971ca8a095583dbfd"
server_key_version: "encryptionkey@0"
I20260811 09:13:23.028923 3435 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 09:13:23.032614 3465 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:13:23.034061 3435 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 09:13:23.034252 3435 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786439250192320-29290-0/minicluster-data/master-0/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786439250192320-29290-0/minicluster-data/master-0/wal
uuid: "31b1cd89c5b144ac82b3af8ae812f770"
format_stamp: "Formatted at 2026-08-11 09:13:23 on dist-test-slave-xt4k"
server_key: "49e8198f3b2593545e55c0cb9f0e8500"
server_key_iv: "1d73d7bad12ba8f971ca8a095583dbfd"
server_key_version: "encryptionkey@0"
I20260811 09:13:23.034459 3435 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786439250192320-29290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786439250192320-29290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786439250192320-29290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:13:23.040798 3435 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:13:23.045362 3435 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:13:23.045673 3435 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:13:23.064002 3435 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36643
I20260811 09:13:23.064082 3576 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36643 every 8 connection(s)
I20260811 09:13:23.065496 3435 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
I20260811 09:13:23.069399 3580 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:13:23.074615 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 3435
I20260811 09:13:23.074950 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786439250192320-29290-0/minicluster-data/master-0/wal/instance
I20260811 09:13:23.075569 29290 external_mini_cluster.cc:1428] Setting key 63c233a5110fb97e747feae1b524af2a
I20260811 09:13:23.081746 3580 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31b1cd89c5b144ac82b3af8ae812f770: Bootstrap starting.
I20260811 09:13:23.084136 3583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:23.077683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54337 (local address 127.0.0.1:36643)
0811 09:13:23.078650 (+ 967us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:23.078663 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:23.078711 (+ 48us) server_negotiation.cc:407] Connection header received
0811 09:13:23.080557 (+ 1846us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:23.080585 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:23.080962 (+ 377us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:23.081349 (+ 387us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:23.082434 (+ 1085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:13:23.082441 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:13:23.082456 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:13:23.082470 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:13:23.082541 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:13:23.082562 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:13:23.082806 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:23.082950 (+ 144us) server_negotiation.cc:299] Negotiation successful
0811 09:13:23.083140 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":696,"thread_start_us":362,"threads_started":1}
I20260811 09:13:23.085644 3580 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 31b1cd89c5b144ac82b3af8ae812f770: Neither blocks nor log segments found. Creating new log.
I20260811 09:13:23.086881 3580 log.cc:824] T 00000000000000000000000000000000 P 31b1cd89c5b144ac82b3af8ae812f770: Log is configured to *not* fsync() on all Append() calls
I20260811 09:13:23.087960 29290 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 09:13:23.090178 3580 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31b1cd89c5b144ac82b3af8ae812f770: No bootstrap required, opened a new log
I20260811 09:13:23.095350 3583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:23.092497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34145 (local address 127.0.0.1:36643)
0811 09:13:23.092723 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:23.092735 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:23.093062 (+ 327us) server_negotiation.cc:407] Connection header received
0811 09:13:23.093232 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:23.093237 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:23.093308 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:23.093438 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:23.094521 (+ 1083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:13:23.094528 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:13:23.094532 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:13:23.094586 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:13:23.094643 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:13:23.094666 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:13:23.094809 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:23.094972 (+ 163us) server_negotiation.cc:299] Negotiation successful
0811 09:13:23.095037 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 09:13:23.099323 3580 raft_consensus.cc:348] T 00000000000000000000000000000000 P 31b1cd89c5b144ac82b3af8ae812f770 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31b1cd89c5b144ac82b3af8ae812f770" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36643 } }
I20260811 09:13:23.099748 3580 raft_consensus.cc:374] T 00000000000000000000000000000000 P 31b1cd89c5b144ac82b3af8ae812f770 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:13:23.099856 3580 raft_consensus.cc:729] T 00000000000000000000000000000000 P 31b1cd89c5b144ac82b3af8ae812f770 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31b1cd89c5b144ac82b3af8ae812f770, State: Initialized, Role: FOLLOWER
I20260811 09:13:23.100507 3580 consensus_queue.cc:260] T 00000000000000000000000000000000 P 31b1cd89c5b144ac82b3af8ae812f770 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31b1cd89c5b144ac82b3af8ae812f770" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36643 } }
I20260811 09:13:23.100773 3580 raft_consensus.cc:388] T 00000000000000000000000000000000 P 31b1cd89c5b144ac82b3af8ae812f770 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:13:23.100916 3580 raft_consensus.cc:482] T 00000000000000000000000000000000 P 31b1cd89c5b144ac82b3af8ae812f770 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:13:23.101089 3580 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 31b1cd89c5b144ac82b3af8ae812f770 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:13:23.103688 3580 raft_consensus.cc:504] T 00000000000000000000000000000000 P 31b1cd89c5b144ac82b3af8ae812f770 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31b1cd89c5b144ac82b3af8ae812f770" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36643 } }
I20260811 09:13:23.104275 3580 leader_election.cc:302] T 00000000000000000000000000000000 P 31b1cd89c5b144ac82b3af8ae812f770 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 31b1cd89c5b144ac82b3af8ae812f770; no voters:
I20260811 09:13:23.104979 3580 leader_election.cc:288] T 00000000000000000000000000000000 P 31b1cd89c5b144ac82b3af8ae812f770 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:13:23.105146 3592 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 31b1cd89c5b144ac82b3af8ae812f770 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:13:23.105852 3592 raft_consensus.cc:686] T 00000000000000000000000000000000 P 31b1cd89c5b144ac82b3af8ae812f770 [term 1 LEADER]: Becoming Leader. State: Replica: 31b1cd89c5b144ac82b3af8ae812f770, State: Running, Role: LEADER
I20260811 09:13:23.106431 3592 consensus_queue.cc:237] T 00000000000000000000000000000000 P 31b1cd89c5b144ac82b3af8ae812f770 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31b1cd89c5b144ac82b3af8ae812f770" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36643 } }
I20260811 09:13:23.107252 3580 sys_catalog.cc:564] T 00000000000000000000000000000000 P 31b1cd89c5b144ac82b3af8ae812f770 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:13:23.112548 3594 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31b1cd89c5b144ac82b3af8ae812f770 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 31b1cd89c5b144ac82b3af8ae812f770. Latest consensus state: current_term: 1 leader_uuid: "31b1cd89c5b144ac82b3af8ae812f770" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31b1cd89c5b144ac82b3af8ae812f770" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36643 } } }
I20260811 09:13:23.113011 3594 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31b1cd89c5b144ac82b3af8ae812f770 [sys.catalog]: This master's current role is: LEADER
I20260811 09:13:23.113689 3593 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31b1cd89c5b144ac82b3af8ae812f770 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "31b1cd89c5b144ac82b3af8ae812f770" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31b1cd89c5b144ac82b3af8ae812f770" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36643 } } }
I20260811 09:13:23.114040 3593 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31b1cd89c5b144ac82b3af8ae812f770 [sys.catalog]: This master's current role is: LEADER
I20260811 09:13:23.114315 3602 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:13:23.120057 3602 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:13:23.130196 3602 catalog_manager.cc:1347] Generated new cluster ID: 228e20bdcb6f4ca9bc80d0c9b159d9fd
I20260811 09:13:23.130330 3602 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:13:23.166493 3602 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:13:23.166980 3602 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:13:23.178598 3602 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 31b1cd89c5b144ac82b3af8ae812f770: Generated new TSK 0
I20260811 09:13:23.179502 3602 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 09:13:23.187932 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 3435
W20260811 09:13:23.189527 3564 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:54337
W20260811 09:13:23.190244 29290 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 3435: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 09:13:23.190400 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 3435
2026-08-11T09:13:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (2212 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786439250192320-29290-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:13:23 dist-test-slave-xt4k krb5kdc[3625](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:13:23 dist-test-slave-xt4k krb5kdc[3625](info): set up 2 sockets
Aug 11 09:13:23 dist-test-slave-xt4k krb5kdc[3625](info): commencing operation
krb5kdc: starting...
W20260811 09:13:24.509248 29290 mini_kdc.cc:121] Time spent starting KDC: real 1.276s user 0.000s sys 0.052s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786439250192320-29290-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786439250192320-29290-0/krb5kdc/test-user.keytab.
Aug 11 09:13:24 dist-test-slave-xt4k krb5kdc[3625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439604, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:13:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:13:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786439250192320-29290-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786439250192320-29290-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 09:13:24.831218 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786439250192320-29290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786439250192320-29290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786439250192320-29290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46355
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786439250192320-29290-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46355
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:13:25.177333 3653 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:13:25.177663 3653 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:13:25.177772 3653 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:13:25.186223 3653 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:13:25.186367 3653 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:13:25.186420 3653 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:13:25.186482 3653 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:13:25.186525 3653 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:13:25.196935 3653 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786439250192320-29290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786439250192320-29290-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46355
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786439250192320-29290-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46355
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786439250192320-29290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:13:25.198280 3653 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:13:25.199565 3653 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:13:25.211373 3661 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:13:25.211349 3665 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:13:25.211315 3662 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:13:25.211841 3653 server_base.cc:1034] running on GCE node
I20260811 09:13:25.212862 3653 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:13:25.214277 3653 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:13:25.215492 3653 hybrid_clock.cc:648] HybridClock initialized: now 1786439605215438 us; error 52 us; skew 500 ppm
I20260811 09:13:25.215893 3653 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:13:25 dist-test-slave-xt4k krb5kdc[3625](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439605, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:13:25.219116 3653 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 09:13:25.220727 3653 webserver.cc:460] Webserver started at http://127.0.0.1:45133/ using document root <none> and password file <none>
I20260811 09:13:25.221410 3653 fs_manager.cc:359] Metadata directory not provided
I20260811 09:13:25.221530 3653 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:13:25.221832 3653 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:13:25.225006 3653 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786439250192320-29290-0/minicluster-data/master-0/data/instance:
uuid: "6bb891adbb4042f4ad24daa2d028ba00"
format_stamp: "Formatted at 2026-08-11 09:13:25 on dist-test-slave-xt4k"
server_key: "ea7bbf1acc825e940fc7e1f14d960417"
server_key_iv: "823e1d2b6cd49c66926f6c4443ad1eae"
server_key_version: "encryptionkey@0"
I20260811 09:13:25.225808 3653 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786439250192320-29290-0/minicluster-data/master-0/wal/instance:
uuid: "6bb891adbb4042f4ad24daa2d028ba00"
format_stamp: "Formatted at 2026-08-11 09:13:25 on dist-test-slave-xt4k"
server_key: "ea7bbf1acc825e940fc7e1f14d960417"
server_key_iv: "823e1d2b6cd49c66926f6c4443ad1eae"
server_key_version: "encryptionkey@0"
I20260811 09:13:25.231191 3653 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 09:13:25.234783 3677 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:13:25.236089 3653 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260811 09:13:25.236258 3653 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786439250192320-29290-0/minicluster-data/master-0/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786439250192320-29290-0/minicluster-data/master-0/wal
uuid: "6bb891adbb4042f4ad24daa2d028ba00"
format_stamp: "Formatted at 2026-08-11 09:13:25 on dist-test-slave-xt4k"
server_key: "ea7bbf1acc825e940fc7e1f14d960417"
server_key_iv: "823e1d2b6cd49c66926f6c4443ad1eae"
server_key_version: "encryptionkey@0"
I20260811 09:13:25.236467 3653 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786439250192320-29290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786439250192320-29290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786439250192320-29290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:13:25.259232 3653 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:13:25.263466 3653 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:13:25.263751 3653 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:13:25.282013 3653 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46355
I20260811 09:13:25.282044 3777 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46355 every 8 connection(s)
I20260811 09:13:25.283454 3653 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
I20260811 09:13:25.287297 3778 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:13:25.293156 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 3653
I20260811 09:13:25.293367 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786439250192320-29290-0/minicluster-data/master-0/wal/instance
I20260811 09:13:25.293921 29290 external_mini_cluster.cc:1428] Setting key c0519530e6a874be25edcbdb67bc2e3d
I20260811 09:13:25.298951 3778 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6bb891adbb4042f4ad24daa2d028ba00: Bootstrap starting.
I20260811 09:13:25.302752 3778 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6bb891adbb4042f4ad24daa2d028ba00: Neither blocks nor log segments found. Creating new log.
I20260811 09:13:25.304004 3778 log.cc:824] T 00000000000000000000000000000000 P 6bb891adbb4042f4ad24daa2d028ba00: Log is configured to *not* fsync() on all Append() calls
I20260811 09:13:25.307020 3778 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6bb891adbb4042f4ad24daa2d028ba00: No bootstrap required, opened a new log
Aug 11 09:13:25 dist-test-slave-xt4k krb5kdc[3625](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439604, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 09:13:25.315804 3778 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6bb891adbb4042f4ad24daa2d028ba00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bb891adbb4042f4ad24daa2d028ba00" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46355 } }
I20260811 09:13:25.316236 3778 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6bb891adbb4042f4ad24daa2d028ba00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:13:25.316368 3778 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6bb891adbb4042f4ad24daa2d028ba00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bb891adbb4042f4ad24daa2d028ba00, State: Initialized, Role: FOLLOWER
I20260811 09:13:25.316987 3778 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6bb891adbb4042f4ad24daa2d028ba00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bb891adbb4042f4ad24daa2d028ba00" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46355 } }
I20260811 09:13:25.317234 3778 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6bb891adbb4042f4ad24daa2d028ba00 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:13:25.317385 3778 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6bb891adbb4042f4ad24daa2d028ba00 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:13:25.317570 3778 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6bb891adbb4042f4ad24daa2d028ba00 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:13:25.319717 3788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:25.296313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.126:51855 (local address 127.0.0.1:46355)
0811 09:13:25.297260 (+ 947us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:25.297273 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:25.297317 (+ 44us) server_negotiation.cc:407] Connection header received
0811 09:13:25.299039 (+ 1722us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:25.299072 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:25.299518 (+ 446us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:25.299856 (+ 338us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:25.301698 (+ 1842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:25.303126 (+ 1428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:25.304925 (+ 1799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:25.305651 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:25.305825 (+ 174us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:25.310193 (+ 4368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:13:25.310204 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:13:25.310226 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:13:25.310275 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:13:25.313552 (+ 3277us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:13:25.315514 (+ 1962us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:13:25.315522 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:13:25.315558 (+ 36us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:13:25.315882 (+ 324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:13:25.316513 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:13:25.316519 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:13:25.316523 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:13:25.317340 (+ 817us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:13:25.317681 (+ 341us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:25.318172 (+ 491us) server_negotiation.cc:299] Negotiation successful
0811 09:13:25.318570 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":696,"thread_start_us":418,"threads_started":1}
I20260811 09:13:25.320307 3778 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6bb891adbb4042f4ad24daa2d028ba00 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bb891adbb4042f4ad24daa2d028ba00" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46355 } }
I20260811 09:13:25.320858 3778 leader_election.cc:302] T 00000000000000000000000000000000 P 6bb891adbb4042f4ad24daa2d028ba00 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6bb891adbb4042f4ad24daa2d028ba00; no voters:
I20260811 09:13:25.321496 3778 leader_election.cc:288] T 00000000000000000000000000000000 P 6bb891adbb4042f4ad24daa2d028ba00 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:13:25.321642 3791 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6bb891adbb4042f4ad24daa2d028ba00 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:13:25.322796 3791 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6bb891adbb4042f4ad24daa2d028ba00 [term 1 LEADER]: Becoming Leader. State: Replica: 6bb891adbb4042f4ad24daa2d028ba00, State: Running, Role: LEADER
I20260811 09:13:25.323403 3791 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6bb891adbb4042f4ad24daa2d028ba00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bb891adbb4042f4ad24daa2d028ba00" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46355 } }
I20260811 09:13:25.323793 3778 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6bb891adbb4042f4ad24daa2d028ba00 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:13:25.324879 29290 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 09:13:25.326628 3792 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6bb891adbb4042f4ad24daa2d028ba00 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6bb891adbb4042f4ad24daa2d028ba00" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bb891adbb4042f4ad24daa2d028ba00" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46355 } } }
I20260811 09:13:25.327064 3792 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6bb891adbb4042f4ad24daa2d028ba00 [sys.catalog]: This master's current role is: LEADER
I20260811 09:13:25.332165 3793 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6bb891adbb4042f4ad24daa2d028ba00 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6bb891adbb4042f4ad24daa2d028ba00. Latest consensus state: current_term: 1 leader_uuid: "6bb891adbb4042f4ad24daa2d028ba00" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6bb891adbb4042f4ad24daa2d028ba00" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46355 } } }
I20260811 09:13:25.334060 3793 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6bb891adbb4042f4ad24daa2d028ba00 [sys.catalog]: This master's current role is: LEADER
I20260811 09:13:25.336735 3800 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:13:25.343865 3800 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:13:25.355890 3827 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 09:13:25.359295 3788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:25.338611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.126:38589 (local address 127.0.0.1:46355)
0811 09:13:25.338800 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:25.338807 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:25.339371 (+ 564us) server_negotiation.cc:407] Connection header received
0811 09:13:25.339691 (+ 320us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:25.339700 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:25.339816 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:25.339898 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:25.342481 (+ 2583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:25.343681 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:25.345608 (+ 1927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:25.346329 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:25.346436 (+ 107us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:25.349831 (+ 3395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:13:25.349838 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:13:25.349843 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:13:25.349886 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:13:25.355553 (+ 5667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:13:25.356921 (+ 1368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:13:25.356932 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:13:25.356934 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:13:25.357098 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:13:25.357631 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:13:25.357635 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:13:25.357637 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:13:25.358041 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:13:25.358290 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:25.358728 (+ 438us) server_negotiation.cc:299] Negotiation successful
0811 09:13:25.358937 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 09:13:25.362919 3800 catalog_manager.cc:1347] Generated new cluster ID: c739d70276cc4f7f916bc3690125688d
I20260811 09:13:25.363054 3800 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:13:25.379827 3800 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:13:25.381321 3800 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:13:25.397737 3800 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6bb891adbb4042f4ad24daa2d028ba00: Generated new TSK 0
I20260811 09:13:25.398916 3800 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 09:13:25.416072 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 3653
I20260811 09:13:25.417445 3770 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:13:25.512290 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 3653
W20260811 09:13:25.535578 3618 connection.cc:535] client connection to 127.0.0.1:46355 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46355): Connection reset by peer (error 104)
2026-08-11T09:13:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (2321 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786439250192320-29290-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:13:25 dist-test-slave-xt4k krb5kdc[3838](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:13:25 dist-test-slave-xt4k krb5kdc[3838](info): set up 2 sockets
Aug 11 09:13:25 dist-test-slave-xt4k krb5kdc[3838](info): commencing operation
krb5kdc: starting...
W20260811 09:13:26.798975 29290 mini_kdc.cc:121] Time spent starting KDC: real 1.245s user 0.002s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786439250192320-29290-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786439250192320-29290-0/krb5kdc/test-user.keytab.
Aug 11 09:13:27 dist-test-slave-xt4k krb5kdc[3838](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439607, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:13:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:13:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786439250192320-29290-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786439250192320-29290-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 09:13:27.145581 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786439250192320-29290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786439250192320-29290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786439250192320-29290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33029
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786439250192320-29290-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33029
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:13:27.491009 3860 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:13:27.491299 3860 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:13:27.491392 3860 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:13:27.498672 3860 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:13:27.498793 3860 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:13:27.498852 3860 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:13:27.498912 3860 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:13:27.498955 3860 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:13:27.509052 3860 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786439250192320-29290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786439250192320-29290-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33029
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786439250192320-29290-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33029
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786439250192320-29290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:13:27.510331 3860 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:13:27.511606 3860 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:13:27.523098 3877 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:13:27.523027 3884 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:13:27.523072 3878 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:13:27.524120 3860 server_base.cc:1034] running on GCE node
I20260811 09:13:27.524940 3860 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:13:27.526373 3860 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:13:27.527622 3860 hybrid_clock.cc:648] HybridClock initialized: now 1786439607527562 us; error 54 us; skew 500 ppm
I20260811 09:13:27.527979 3860 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:13:27 dist-test-slave-xt4k krb5kdc[3838](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439607, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:13:27.531313 3860 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 09:13:27.532903 3860 webserver.cc:460] Webserver started at http://127.0.0.1:33347/ using document root <none> and password file <none>
I20260811 09:13:27.533614 3860 fs_manager.cc:359] Metadata directory not provided
I20260811 09:13:27.533818 3860 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:13:27.534168 3860 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:13:27.537382 3860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786439250192320-29290-0/minicluster-data/master-0/data/instance:
uuid: "ef0aa9aae66c4a358002454bd586da43"
format_stamp: "Formatted at 2026-08-11 09:13:27 on dist-test-slave-xt4k"
server_key: "f2e19e2710806c854c1cec95f7773ffa"
server_key_iv: "f868ee5da4ea86d8e9f98f1a2a1f09fa"
server_key_version: "encryptionkey@0"
I20260811 09:13:27.538247 3860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786439250192320-29290-0/minicluster-data/master-0/wal/instance:
uuid: "ef0aa9aae66c4a358002454bd586da43"
format_stamp: "Formatted at 2026-08-11 09:13:27 on dist-test-slave-xt4k"
server_key: "f2e19e2710806c854c1cec95f7773ffa"
server_key_iv: "f868ee5da4ea86d8e9f98f1a2a1f09fa"
server_key_version: "encryptionkey@0"
I20260811 09:13:27.543658 3860 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 09:13:27.547310 3897 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:13:27.548770 3860 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 09:13:27.548936 3860 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786439250192320-29290-0/minicluster-data/master-0/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786439250192320-29290-0/minicluster-data/master-0/wal
uuid: "ef0aa9aae66c4a358002454bd586da43"
format_stamp: "Formatted at 2026-08-11 09:13:27 on dist-test-slave-xt4k"
server_key: "f2e19e2710806c854c1cec95f7773ffa"
server_key_iv: "f868ee5da4ea86d8e9f98f1a2a1f09fa"
server_key_version: "encryptionkey@0"
I20260811 09:13:27.549149 3860 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786439250192320-29290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786439250192320-29290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786439250192320-29290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:13:27.583235 3860 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:13:27.587703 3860 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:13:27.588029 3860 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:13:27.606206 3860 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33029
I20260811 09:13:27.606251 3995 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33029 every 8 connection(s)
I20260811 09:13:27.607586 3860 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
I20260811 09:13:27.609014 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 3860
I20260811 09:13:27.609179 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786439250192320-29290-0/minicluster-data/master-0/wal/instance
I20260811 09:13:27.609603 29290 external_mini_cluster.cc:1428] Setting key d8cbb40d3aaa46af6636c6bfdd5d15d0
I20260811 09:13:27.611732 3997 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:13:27.624650 4001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:27.611464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.126:50875 (local address 127.0.0.1:33029)
0811 09:13:27.612245 (+ 781us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:27.612260 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:27.612309 (+ 49us) server_negotiation.cc:407] Connection header received
0811 09:13:27.614513 (+ 2204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:27.614543 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:27.615054 (+ 511us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:27.615570 (+ 516us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:27.616084 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:27.618126 (+ 2042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:27.619390 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:27.620430 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:27.620622 (+ 192us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:27.621784 (+ 1162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:13:27.621795 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:13:27.621817 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:13:27.622566 (+ 749us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:13:27.622660 (+ 94us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:13:27.622701 (+ 41us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:13:27.622849 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:27.623133 (+ 284us) server_negotiation.cc:299] Negotiation successful
0811 09:13:27.623317 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":547,"thread_start_us":305,"threads_started":1}
I20260811 09:13:27.625995 3997 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef0aa9aae66c4a358002454bd586da43: Bootstrap starting.
I20260811 09:13:27.628638 29290 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 09:13:27.629313 3997 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ef0aa9aae66c4a358002454bd586da43: Neither blocks nor log segments found. Creating new log.
I20260811 09:13:27.630259 3997 log.cc:824] T 00000000000000000000000000000000 P ef0aa9aae66c4a358002454bd586da43: Log is configured to *not* fsync() on all Append() calls
I20260811 09:13:27.633322 3997 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef0aa9aae66c4a358002454bd586da43: No bootstrap required, opened a new log
I20260811 09:13:27.638952 4001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:27.632830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.126:54055 (local address 127.0.0.1:33029)
0811 09:13:27.632974 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:27.632978 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:27.633424 (+ 446us) server_negotiation.cc:407] Connection header received
0811 09:13:27.633597 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:27.633603 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:27.633667 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:27.633763 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:27.634250 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:27.635090 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:27.636471 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:27.636978 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:27.637102 (+ 124us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:27.638225 (+ 1123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:13:27.638230 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:13:27.638233 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:13:27.638251 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:13:27.638293 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:13:27.638309 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:13:27.638409 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:27.638619 (+ 210us) server_negotiation.cc:299] Negotiation successful
0811 09:13:27.638670 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 09:13:27.641772 3997 raft_consensus.cc:348] T 00000000000000000000000000000000 P ef0aa9aae66c4a358002454bd586da43 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef0aa9aae66c4a358002454bd586da43" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33029 } }
I20260811 09:13:27.642132 3997 raft_consensus.cc:374] T 00000000000000000000000000000000 P ef0aa9aae66c4a358002454bd586da43 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:13:27.642225 3997 raft_consensus.cc:729] T 00000000000000000000000000000000 P ef0aa9aae66c4a358002454bd586da43 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef0aa9aae66c4a358002454bd586da43, State: Initialized, Role: FOLLOWER
I20260811 09:13:27.642807 3997 consensus_queue.cc:260] T 00000000000000000000000000000000 P ef0aa9aae66c4a358002454bd586da43 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef0aa9aae66c4a358002454bd586da43" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33029 } }
I20260811 09:13:27.643033 3997 raft_consensus.cc:388] T 00000000000000000000000000000000 P ef0aa9aae66c4a358002454bd586da43 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:13:27.643157 3997 raft_consensus.cc:482] T 00000000000000000000000000000000 P ef0aa9aae66c4a358002454bd586da43 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:13:27.643312 3997 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ef0aa9aae66c4a358002454bd586da43 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:13:27.645895 3997 raft_consensus.cc:504] T 00000000000000000000000000000000 P ef0aa9aae66c4a358002454bd586da43 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef0aa9aae66c4a358002454bd586da43" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33029 } }
I20260811 09:13:27.646377 3997 leader_election.cc:302] T 00000000000000000000000000000000 P ef0aa9aae66c4a358002454bd586da43 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ef0aa9aae66c4a358002454bd586da43; no voters:
I20260811 09:13:27.646914 3997 leader_election.cc:288] T 00000000000000000000000000000000 P ef0aa9aae66c4a358002454bd586da43 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:13:27.647051 4025 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ef0aa9aae66c4a358002454bd586da43 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:13:27.647742 4025 raft_consensus.cc:686] T 00000000000000000000000000000000 P ef0aa9aae66c4a358002454bd586da43 [term 1 LEADER]: Becoming Leader. State: Replica: ef0aa9aae66c4a358002454bd586da43, State: Running, Role: LEADER
I20260811 09:13:27.648301 4025 consensus_queue.cc:237] T 00000000000000000000000000000000 P ef0aa9aae66c4a358002454bd586da43 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef0aa9aae66c4a358002454bd586da43" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33029 } }
I20260811 09:13:27.648887 3997 sys_catalog.cc:564] T 00000000000000000000000000000000 P ef0aa9aae66c4a358002454bd586da43 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:13:27.651036 4027 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef0aa9aae66c4a358002454bd586da43 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ef0aa9aae66c4a358002454bd586da43. Latest consensus state: current_term: 1 leader_uuid: "ef0aa9aae66c4a358002454bd586da43" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef0aa9aae66c4a358002454bd586da43" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33029 } } }
I20260811 09:13:27.651443 4027 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef0aa9aae66c4a358002454bd586da43 [sys.catalog]: This master's current role is: LEADER
I20260811 09:13:27.651698 4026 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef0aa9aae66c4a358002454bd586da43 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ef0aa9aae66c4a358002454bd586da43" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef0aa9aae66c4a358002454bd586da43" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33029 } } }
I20260811 09:13:27.652074 4026 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef0aa9aae66c4a358002454bd586da43 [sys.catalog]: This master's current role is: LEADER
I20260811 09:13:27.653342 4033 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:13:27.657758 4033 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:13:27.666438 4033 catalog_manager.cc:1347] Generated new cluster ID: ff7fe4f4cdda4f7c9a770817b26d299e
I20260811 09:13:27.666554 4033 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:13:27.676306 4033 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:13:27.677691 4033 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:13:27.689620 4033 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ef0aa9aae66c4a358002454bd586da43: Generated new TSK 0
I20260811 09:13:27.690557 4033 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 09:13:27.697810 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 3860
W20260811 09:13:27.699494 3990 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.126:50875
W20260811 09:13:27.700433 29290 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 3860: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 09:13:27.700628 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 3860
W20260811 09:13:27.728116 3835 connection.cc:535] client connection to 127.0.0.1:33029 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33029): Connection reset by peer (error 104)
2026-08-11T09:13:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (2192 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786439250192320-29290-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:13:27 dist-test-slave-xt4k krb5kdc[4058](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:13:27 dist-test-slave-xt4k krb5kdc[4058](info): set up 2 sockets
Aug 11 09:13:27 dist-test-slave-xt4k krb5kdc[4058](info): commencing operation
krb5kdc: starting...
W20260811 09:13:29.003537 29290 mini_kdc.cc:121] Time spent starting KDC: real 1.256s user 0.000s sys 0.049s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786439250192320-29290-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786439250192320-29290-0/krb5kdc/test-user.keytab.
Aug 11 09:13:29 dist-test-slave-xt4k krb5kdc[4058](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439609, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:13:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:13:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786439250192320-29290-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786439250192320-29290-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 09:13:29.335680 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786439250192320-29290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786439250192320-29290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786439250192320-29290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42567
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786439250192320-29290-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37883
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42567
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:13:29.682446 4079 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:13:29.682760 4079 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:13:29.682854 4079 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:13:29.690290 4079 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:13:29.690430 4079 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:13:29.690487 4079 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:13:29.690541 4079 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:13:29.690593 4079 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:13:29.700796 4079 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37883
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786439250192320-29290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786439250192320-29290-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42567
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786439250192320-29290-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42567
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786439250192320-29290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:13:29.702095 4079 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:13:29.703409 4079 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:13:29.715785 4089 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:13:29.715764 4094 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:13:29.715833 4090 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:13:29.716653 4079 server_base.cc:1034] running on GCE node
I20260811 09:13:29.717514 4079 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:13:29.719071 4079 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:13:29.720302 4079 hybrid_clock.cc:648] HybridClock initialized: now 1786439609720257 us; error 46 us; skew 500 ppm
I20260811 09:13:29.720753 4079 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:13:29 dist-test-slave-xt4k krb5kdc[4058](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439609, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:13:29.724159 4079 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 09:13:29.725899 4079 webserver.cc:460] Webserver started at http://127.0.0.1:44117/ using document root <none> and password file <none>
I20260811 09:13:29.726593 4079 fs_manager.cc:359] Metadata directory not provided
I20260811 09:13:29.726706 4079 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:13:29.726976 4079 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:13:29.730123 4079 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786439250192320-29290-0/minicluster-data/master-0/data/instance:
uuid: "233c37ce88e2448db8819da1c2fca5d1"
format_stamp: "Formatted at 2026-08-11 09:13:29 on dist-test-slave-xt4k"
server_key: "41391beb43ab73c46f93235e0d8208f9"
server_key_iv: "ea4a644261deb70d5b421d9c8571d755"
server_key_version: "encryptionkey@0"
I20260811 09:13:29.730867 4079 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786439250192320-29290-0/minicluster-data/master-0/wal/instance:
uuid: "233c37ce88e2448db8819da1c2fca5d1"
format_stamp: "Formatted at 2026-08-11 09:13:29 on dist-test-slave-xt4k"
server_key: "41391beb43ab73c46f93235e0d8208f9"
server_key_iv: "ea4a644261deb70d5b421d9c8571d755"
server_key_version: "encryptionkey@0"
I20260811 09:13:29.736565 4079 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 09:13:29.740166 4110 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:13:29.741693 4079 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 09:13:29.741859 4079 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786439250192320-29290-0/minicluster-data/master-0/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786439250192320-29290-0/minicluster-data/master-0/wal
uuid: "233c37ce88e2448db8819da1c2fca5d1"
format_stamp: "Formatted at 2026-08-11 09:13:29 on dist-test-slave-xt4k"
server_key: "41391beb43ab73c46f93235e0d8208f9"
server_key_iv: "ea4a644261deb70d5b421d9c8571d755"
server_key_version: "encryptionkey@0"
I20260811 09:13:29.742045 4079 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786439250192320-29290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786439250192320-29290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786439250192320-29290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:13:29.748055 4079 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:13:29.752589 4079 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:13:29.752862 4079 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:13:29.771286 4079 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42567
I20260811 09:13:29.771341 4198 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42567 every 8 connection(s)
I20260811 09:13:29.772719 4079 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
I20260811 09:13:29.776961 4199 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:13:29.777447 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 4079
I20260811 09:13:29.777596 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786439250192320-29290-0/minicluster-data/master-0/wal/instance
I20260811 09:13:29.778038 29290 external_mini_cluster.cc:1428] Setting key 6b1331c1698159ee45b9097427a822d3
I20260811 09:13:29.787431 4204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:29.779935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.126:55637 (local address 127.0.0.1:42567)
0811 09:13:29.780790 (+ 855us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:29.780810 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:29.780862 (+ 52us) server_negotiation.cc:407] Connection header received
0811 09:13:29.783517 (+ 2655us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:29.783553 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:29.784032 (+ 479us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:29.784530 (+ 498us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:29.785467 (+ 937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:13:29.785486 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:13:29.785504 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:13:29.785524 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:13:29.785596 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:13:29.785630 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:13:29.785853 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:29.785982 (+ 129us) server_negotiation.cc:299] Negotiation successful
0811 09:13:29.786190 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":579,"thread_start_us":337,"threads_started":1}
I20260811 09:13:29.791313 29290 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 09:13:29.791697 4199 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 233c37ce88e2448db8819da1c2fca5d1: Bootstrap starting.
I20260811 09:13:29.794930 4199 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 233c37ce88e2448db8819da1c2fca5d1: Neither blocks nor log segments found. Creating new log.
I20260811 09:13:29.796262 4199 log.cc:824] T 00000000000000000000000000000000 P 233c37ce88e2448db8819da1c2fca5d1: Log is configured to *not* fsync() on all Append() calls
I20260811 09:13:29.798226 4204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:29.795675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.126:44343 (local address 127.0.0.1:42567)
0811 09:13:29.795821 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:29.795826 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:29.796380 (+ 554us) server_negotiation.cc:407] Connection header received
0811 09:13:29.796575 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:29.796579 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:29.796630 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:29.796713 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:29.797607 (+ 894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:13:29.797614 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:13:29.797619 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:13:29.797636 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:13:29.797676 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:13:29.797691 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:13:29.797789 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:29.797890 (+ 101us) server_negotiation.cc:299] Negotiation successful
0811 09:13:29.797957 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 09:13:29.799409 4199 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 233c37ce88e2448db8819da1c2fca5d1: No bootstrap required, opened a new log
I20260811 09:13:29.806834 4199 raft_consensus.cc:348] T 00000000000000000000000000000000 P 233c37ce88e2448db8819da1c2fca5d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "233c37ce88e2448db8819da1c2fca5d1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42567 } }
I20260811 09:13:29.807209 4199 raft_consensus.cc:374] T 00000000000000000000000000000000 P 233c37ce88e2448db8819da1c2fca5d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:13:29.807303 4199 raft_consensus.cc:729] T 00000000000000000000000000000000 P 233c37ce88e2448db8819da1c2fca5d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 233c37ce88e2448db8819da1c2fca5d1, State: Initialized, Role: FOLLOWER
I20260811 09:13:29.807847 4199 consensus_queue.cc:260] T 00000000000000000000000000000000 P 233c37ce88e2448db8819da1c2fca5d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "233c37ce88e2448db8819da1c2fca5d1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42567 } }
I20260811 09:13:29.808074 4199 raft_consensus.cc:388] T 00000000000000000000000000000000 P 233c37ce88e2448db8819da1c2fca5d1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:13:29.808200 4199 raft_consensus.cc:482] T 00000000000000000000000000000000 P 233c37ce88e2448db8819da1c2fca5d1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:13:29.808411 4199 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 233c37ce88e2448db8819da1c2fca5d1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:13:29.810874 4199 raft_consensus.cc:504] T 00000000000000000000000000000000 P 233c37ce88e2448db8819da1c2fca5d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "233c37ce88e2448db8819da1c2fca5d1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42567 } }
I20260811 09:13:29.811393 4199 leader_election.cc:302] T 00000000000000000000000000000000 P 233c37ce88e2448db8819da1c2fca5d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 233c37ce88e2448db8819da1c2fca5d1; no voters:
I20260811 09:13:29.811995 4199 leader_election.cc:288] T 00000000000000000000000000000000 P 233c37ce88e2448db8819da1c2fca5d1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:13:29.812168 4212 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 233c37ce88e2448db8819da1c2fca5d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:13:29.812801 4212 raft_consensus.cc:686] T 00000000000000000000000000000000 P 233c37ce88e2448db8819da1c2fca5d1 [term 1 LEADER]: Becoming Leader. State: Replica: 233c37ce88e2448db8819da1c2fca5d1, State: Running, Role: LEADER
I20260811 09:13:29.813418 4212 consensus_queue.cc:237] T 00000000000000000000000000000000 P 233c37ce88e2448db8819da1c2fca5d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "233c37ce88e2448db8819da1c2fca5d1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42567 } }
I20260811 09:13:29.814227 4199 sys_catalog.cc:564] T 00000000000000000000000000000000 P 233c37ce88e2448db8819da1c2fca5d1 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:13:29.816558 4213 sys_catalog.cc:455] T 00000000000000000000000000000000 P 233c37ce88e2448db8819da1c2fca5d1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "233c37ce88e2448db8819da1c2fca5d1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "233c37ce88e2448db8819da1c2fca5d1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42567 } } }
I20260811 09:13:29.816622 4214 sys_catalog.cc:455] T 00000000000000000000000000000000 P 233c37ce88e2448db8819da1c2fca5d1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 233c37ce88e2448db8819da1c2fca5d1. Latest consensus state: current_term: 1 leader_uuid: "233c37ce88e2448db8819da1c2fca5d1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "233c37ce88e2448db8819da1c2fca5d1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42567 } } }
I20260811 09:13:29.817189 4214 sys_catalog.cc:458] T 00000000000000000000000000000000 P 233c37ce88e2448db8819da1c2fca5d1 [sys.catalog]: This master's current role is: LEADER
I20260811 09:13:29.817191 4213 sys_catalog.cc:458] T 00000000000000000000000000000000 P 233c37ce88e2448db8819da1c2fca5d1 [sys.catalog]: This master's current role is: LEADER
I20260811 09:13:29.818001 4230 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:13:29.825357 4230 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:13:29.835286 4230 catalog_manager.cc:1347] Generated new cluster ID: e865422d6b2f46b4be1262054466262f
I20260811 09:13:29.835412 4230 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:13:29.849927 4230 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:13:29.850419 4230 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:13:29.861812 4230 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 233c37ce88e2448db8819da1c2fca5d1: Generated new TSK 0
I20260811 09:13:29.862823 4230 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 09:13:29.881405 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 4079
W20260811 09:13:29.883104 4193 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.126:55637
W20260811 09:13:29.883939 29290 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 4079: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 09:13:29.884092 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 4079
2026-08-11T09:13:29Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (2184 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786439250192320-29290-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:13:30 dist-test-slave-xt4k krb5kdc[4249](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:13:30 dist-test-slave-xt4k krb5kdc[4249](info): set up 2 sockets
Aug 11 09:13:30 dist-test-slave-xt4k krb5kdc[4249](info): commencing operation
krb5kdc: starting...
W20260811 09:13:31.197456 29290 mini_kdc.cc:121] Time spent starting KDC: real 1.266s user 0.000s sys 0.050s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786439250192320-29290-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786439250192320-29290-0/krb5kdc/test-user.keytab.
Aug 11 09:13:31 dist-test-slave-xt4k krb5kdc[4249](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439611, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:13:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:13:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786439250192320-29290-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786439250192320-29290-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 09:13:31.529412 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786439250192320-29290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786439250192320-29290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786439250192320-29290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35159
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786439250192320-29290-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35159
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:13:31.876749 4275 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:13:31.877053 4275 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:13:31.877195 4275 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:13:31.884488 4275 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:13:31.884606 4275 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:13:31.884677 4275 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:13:31.884770 4275 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:13:31.884835 4275 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:13:31.894750 4275 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786439250192320-29290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786439250192320-29290-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35159
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786439250192320-29290-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35159
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786439250192320-29290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:13:31.896082 4275 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:13:31.897411 4275 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:13:31.908912 4286 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:13:31.908981 4282 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:13:31.908977 4289 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:13:31.909838 4275 server_base.cc:1034] running on GCE node
I20260811 09:13:31.910627 4275 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:13:31.912103 4275 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:13:31.913301 4275 hybrid_clock.cc:648] HybridClock initialized: now 1786439611913244 us; error 48 us; skew 500 ppm
I20260811 09:13:31.913694 4275 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:13:31 dist-test-slave-xt4k krb5kdc[4249](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439611, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:13:31.919688 4275 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 09:13:31.921212 4275 webserver.cc:460] Webserver started at http://127.0.0.1:45415/ using document root <none> and password file <none>
I20260811 09:13:31.921906 4275 fs_manager.cc:359] Metadata directory not provided
I20260811 09:13:31.922013 4275 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:13:31.922266 4275 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:13:31.925380 4275 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786439250192320-29290-0/minicluster-data/master-0/data/instance:
uuid: "d5ee5cdeafee4b9b87d78b89594598f4"
format_stamp: "Formatted at 2026-08-11 09:13:31 on dist-test-slave-xt4k"
server_key: "db499edf38a4767cc7d74d90fe0ff1df"
server_key_iv: "e6bd0bbc33819b7d53b306625644834f"
server_key_version: "encryptionkey@0"
I20260811 09:13:31.926151 4275 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786439250192320-29290-0/minicluster-data/master-0/wal/instance:
uuid: "d5ee5cdeafee4b9b87d78b89594598f4"
format_stamp: "Formatted at 2026-08-11 09:13:31 on dist-test-slave-xt4k"
server_key: "db499edf38a4767cc7d74d90fe0ff1df"
server_key_iv: "e6bd0bbc33819b7d53b306625644834f"
server_key_version: "encryptionkey@0"
I20260811 09:13:31.931530 4275 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 09:13:31.935220 4298 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:13:31.936673 4275 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 09:13:31.936858 4275 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786439250192320-29290-0/minicluster-data/master-0/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786439250192320-29290-0/minicluster-data/master-0/wal
uuid: "d5ee5cdeafee4b9b87d78b89594598f4"
format_stamp: "Formatted at 2026-08-11 09:13:31 on dist-test-slave-xt4k"
server_key: "db499edf38a4767cc7d74d90fe0ff1df"
server_key_iv: "e6bd0bbc33819b7d53b306625644834f"
server_key_version: "encryptionkey@0"
I20260811 09:13:31.937068 4275 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786439250192320-29290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786439250192320-29290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786439250192320-29290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:13:31.952916 4275 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:13:31.957157 4275 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:13:31.957444 4275 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:13:31.975323 4275 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35159
I20260811 09:13:31.975354 4383 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35159 every 8 connection(s)
I20260811 09:13:31.976694 4275 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
I20260811 09:13:31.980439 4385 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:13:31.982220 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 4275
I20260811 09:13:31.982411 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786439250192320-29290-0/minicluster-data/master-0/wal/instance
I20260811 09:13:31.982956 29290 external_mini_cluster.cc:1428] Setting key f163b4f5128e5c56edfd67bad425dbf5
I20260811 09:13:31.994515 4385 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d5ee5cdeafee4b9b87d78b89594598f4: Bootstrap starting.
Aug 11 09:13:31 dist-test-slave-xt4k krb5kdc[4249](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439611, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 09:13:31.998427 4385 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d5ee5cdeafee4b9b87d78b89594598f4: Neither blocks nor log segments found. Creating new log.
I20260811 09:13:31.999634 4385 log.cc:824] T 00000000000000000000000000000000 P d5ee5cdeafee4b9b87d78b89594598f4: Log is configured to *not* fsync() on all Append() calls
I20260811 09:13:32.003197 4385 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d5ee5cdeafee4b9b87d78b89594598f4: No bootstrap required, opened a new log
I20260811 09:13:32.007164 4392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:31.984784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.126:49111 (local address 127.0.0.1:35159)
0811 09:13:31.985573 (+ 789us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:31.985588 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:31.985636 (+ 48us) server_negotiation.cc:407] Connection header received
0811 09:13:31.987180 (+ 1544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:31.987207 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:31.987615 (+ 408us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:31.987927 (+ 312us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:31.989797 (+ 1870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:31.991138 (+ 1341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:31.992883 (+ 1745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:31.993753 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:31.993926 (+ 173us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:31.999008 (+ 5082us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:13:31.999017 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:13:31.999034 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:13:31.999076 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:13:32.002355 (+ 3279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:13:32.003453 (+ 1098us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:13:32.003462 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:13:32.003469 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:13:32.003670 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:13:32.004266 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:13:32.004270 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:13:32.004272 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:13:32.004887 (+ 615us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:13:32.005170 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:32.005727 (+ 557us) server_negotiation.cc:299] Negotiation successful
0811 09:13:32.006095 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":574,"thread_start_us":350,"threads_started":1}
I20260811 09:13:32.011655 29290 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 09:13:32.012754 4385 raft_consensus.cc:348] T 00000000000000000000000000000000 P d5ee5cdeafee4b9b87d78b89594598f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5ee5cdeafee4b9b87d78b89594598f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35159 } }
I20260811 09:13:32.013155 4385 raft_consensus.cc:374] T 00000000000000000000000000000000 P d5ee5cdeafee4b9b87d78b89594598f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:13:32.013262 4385 raft_consensus.cc:729] T 00000000000000000000000000000000 P d5ee5cdeafee4b9b87d78b89594598f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5ee5cdeafee4b9b87d78b89594598f4, State: Initialized, Role: FOLLOWER
I20260811 09:13:32.013886 4385 consensus_queue.cc:260] T 00000000000000000000000000000000 P d5ee5cdeafee4b9b87d78b89594598f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5ee5cdeafee4b9b87d78b89594598f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35159 } }
I20260811 09:13:32.014151 4385 raft_consensus.cc:388] T 00000000000000000000000000000000 P d5ee5cdeafee4b9b87d78b89594598f4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:13:32.014295 4385 raft_consensus.cc:482] T 00000000000000000000000000000000 P d5ee5cdeafee4b9b87d78b89594598f4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:13:32.014482 4385 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d5ee5cdeafee4b9b87d78b89594598f4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:13:32.017649 4385 raft_consensus.cc:504] T 00000000000000000000000000000000 P d5ee5cdeafee4b9b87d78b89594598f4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5ee5cdeafee4b9b87d78b89594598f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35159 } }
I20260811 09:13:32.018201 4385 leader_election.cc:302] T 00000000000000000000000000000000 P d5ee5cdeafee4b9b87d78b89594598f4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d5ee5cdeafee4b9b87d78b89594598f4; no voters:
I20260811 09:13:32.018816 4385 leader_election.cc:288] T 00000000000000000000000000000000 P d5ee5cdeafee4b9b87d78b89594598f4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:13:32.018947 4402 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d5ee5cdeafee4b9b87d78b89594598f4 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:13:32.020458 4402 raft_consensus.cc:686] T 00000000000000000000000000000000 P d5ee5cdeafee4b9b87d78b89594598f4 [term 1 LEADER]: Becoming Leader. State: Replica: d5ee5cdeafee4b9b87d78b89594598f4, State: Running, Role: LEADER
I20260811 09:13:32.021152 4402 consensus_queue.cc:237] T 00000000000000000000000000000000 P d5ee5cdeafee4b9b87d78b89594598f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5ee5cdeafee4b9b87d78b89594598f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35159 } }
I20260811 09:13:32.021449 4385 sys_catalog.cc:564] T 00000000000000000000000000000000 P d5ee5cdeafee4b9b87d78b89594598f4 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:13:32.024597 4404 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5ee5cdeafee4b9b87d78b89594598f4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d5ee5cdeafee4b9b87d78b89594598f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5ee5cdeafee4b9b87d78b89594598f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35159 } } }
I20260811 09:13:32.024984 4404 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5ee5cdeafee4b9b87d78b89594598f4 [sys.catalog]: This master's current role is: LEADER
I20260811 09:13:32.024278 4408 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5ee5cdeafee4b9b87d78b89594598f4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d5ee5cdeafee4b9b87d78b89594598f4. Latest consensus state: current_term: 1 leader_uuid: "d5ee5cdeafee4b9b87d78b89594598f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5ee5cdeafee4b9b87d78b89594598f4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35159 } } }
I20260811 09:13:32.028937 4408 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5ee5cdeafee4b9b87d78b89594598f4 [sys.catalog]: This master's current role is: LEADER
I20260811 09:13:32.036574 4412 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:13:32.037519 4392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:32.016303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.126:44133 (local address 127.0.0.1:35159)
0811 09:13:32.016536 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:32.016544 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:32.016913 (+ 369us) server_negotiation.cc:407] Connection header received
0811 09:13:32.017093 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:32.017099 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:32.017166 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:32.017256 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:32.018871 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:32.020081 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:32.021569 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:32.022017 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:32.022135 (+ 118us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:32.028617 (+ 6482us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:13:32.028628 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:13:32.028634 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:13:32.028682 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:13:32.032962 (+ 4280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:13:32.034298 (+ 1336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:13:32.034304 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:13:32.034308 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:13:32.034509 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:13:32.035147 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:13:32.035153 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:13:32.035157 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:13:32.035786 (+ 629us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:13:32.036711 (+ 925us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:32.036928 (+ 217us) server_negotiation.cc:299] Negotiation successful
0811 09:13:32.037121 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260811 09:13:32.043248 4412 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:13:32.052662 4412 catalog_manager.cc:1347] Generated new cluster ID: 99c3cbd0360e4a118aecffa377d4bddb
I20260811 09:13:32.052800 4412 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:13:32.075781 4412 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:13:32.077260 4412 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:13:32.089877 4412 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d5ee5cdeafee4b9b87d78b89594598f4: Generated new TSK 0
I20260811 09:13:32.091053 4412 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 09:13:32.102479 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 4275
I20260811 09:13:32.103703 4379 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:13:32.197213 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 4275
W20260811 09:13:32.222636 4244 connection.cc:535] client connection to 127.0.0.1:35159 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35159): Connection reset by peer (error 104)
2026-08-11T09:13:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (2309 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786439250192320-29290-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:13:32 dist-test-slave-xt4k krb5kdc[4446](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:13:32 dist-test-slave-xt4k krb5kdc[4446](info): set up 2 sockets
Aug 11 09:13:32 dist-test-slave-xt4k krb5kdc[4446](info): commencing operation
krb5kdc: starting...
W20260811 09:13:33.503055 29290 mini_kdc.cc:121] Time spent starting KDC: real 1.261s user 0.002s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786439250192320-29290-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786439250192320-29290-0/krb5kdc/test-user.keytab.
Aug 11 09:13:33 dist-test-slave-xt4k krb5kdc[4446](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439613, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:13:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:13:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786439250192320-29290-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786439250192320-29290-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 09:13:33.836544 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786439250192320-29290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786439250192320-29290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786439250192320-29290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34247
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786439250192320-29290-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34247
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:13:34.183894 4462 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:13:34.184199 4462 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:13:34.184329 4462 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:13:34.191823 4462 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:13:34.191957 4462 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:13:34.192010 4462 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:13:34.192108 4462 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:13:34.192159 4462 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:13:34.202504 4462 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786439250192320-29290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786439250192320-29290-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34247
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786439250192320-29290-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34247
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786439250192320-29290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:13:34.204016 4462 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:13:34.205494 4462 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:13:34.218343 4470 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:13:34.218367 4471 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:13:34.218979 4475 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:13:34.219209 4462 server_base.cc:1034] running on GCE node
I20260811 09:13:34.219997 4462 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:13:34.221510 4462 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:13:34.222731 4462 hybrid_clock.cc:648] HybridClock initialized: now 1786439614222675 us; error 60 us; skew 500 ppm
I20260811 09:13:34.223098 4462 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:13:34 dist-test-slave-xt4k krb5kdc[4446](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439614, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:13:34.226413 4462 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 09:13:34.227981 4462 webserver.cc:460] Webserver started at http://127.0.0.1:41485/ using document root <none> and password file <none>
I20260811 09:13:34.228788 4462 fs_manager.cc:359] Metadata directory not provided
I20260811 09:13:34.228915 4462 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:13:34.229212 4462 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:13:34.232699 4462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786439250192320-29290-0/minicluster-data/master-0/data/instance:
uuid: "cbbde7f6994a4afaafe4eb3ba0114975"
format_stamp: "Formatted at 2026-08-11 09:13:34 on dist-test-slave-xt4k"
server_key: "f7578a8c6d3306e2211bd42125ef9b1f"
server_key_iv: "7256483c1330347bfb21296a28c68c62"
server_key_version: "encryptionkey@0"
I20260811 09:13:34.233494 4462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786439250192320-29290-0/minicluster-data/master-0/wal/instance:
uuid: "cbbde7f6994a4afaafe4eb3ba0114975"
format_stamp: "Formatted at 2026-08-11 09:13:34 on dist-test-slave-xt4k"
server_key: "f7578a8c6d3306e2211bd42125ef9b1f"
server_key_iv: "7256483c1330347bfb21296a28c68c62"
server_key_version: "encryptionkey@0"
I20260811 09:13:34.238968 4462 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 09:13:34.242852 4490 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:13:34.244570 4462 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 09:13:34.244763 4462 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786439250192320-29290-0/minicluster-data/master-0/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786439250192320-29290-0/minicluster-data/master-0/wal
uuid: "cbbde7f6994a4afaafe4eb3ba0114975"
format_stamp: "Formatted at 2026-08-11 09:13:34 on dist-test-slave-xt4k"
server_key: "f7578a8c6d3306e2211bd42125ef9b1f"
server_key_iv: "7256483c1330347bfb21296a28c68c62"
server_key_version: "encryptionkey@0"
I20260811 09:13:34.244971 4462 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786439250192320-29290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786439250192320-29290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786439250192320-29290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:13:34.260831 4462 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:13:34.265578 4462 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:13:34.265872 4462 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:13:34.287313 4462 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34247
I20260811 09:13:34.287370 4608 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34247 every 8 connection(s)
I20260811 09:13:34.289005 4462 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
I20260811 09:13:34.289535 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 4462
I20260811 09:13:34.289812 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786439250192320-29290-0/minicluster-data/master-0/wal/instance
I20260811 09:13:34.290444 29290 external_mini_cluster.cc:1428] Setting key dd7da0a647192cc80b31fe0b0fc5b135
I20260811 09:13:34.294214 4609 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:13:34.306375 4612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:34.292471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.126:42473 (local address 127.0.0.1:34247)
0811 09:13:34.293741 (+ 1270us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:34.293755 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:34.293802 (+ 47us) server_negotiation.cc:407] Connection header received
0811 09:13:34.295794 (+ 1992us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:34.295822 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:34.297105 (+ 1283us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:34.297768 (+ 663us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:34.298441 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:34.299946 (+ 1505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:34.301372 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:34.302117 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:34.302308 (+ 191us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:34.303812 (+ 1504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:13:34.303825 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:13:34.303844 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:13:34.304524 (+ 680us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:13:34.304618 (+ 94us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:13:34.304654 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:13:34.304834 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:34.305154 (+ 320us) server_negotiation.cc:299] Negotiation successful
0811 09:13:34.305336 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":676,"thread_start_us":453,"threads_started":1}
I20260811 09:13:34.307279 4609 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cbbde7f6994a4afaafe4eb3ba0114975: Bootstrap starting.
I20260811 09:13:34.310570 29290 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 09:13:34.310883 4609 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cbbde7f6994a4afaafe4eb3ba0114975: Neither blocks nor log segments found. Creating new log.
I20260811 09:13:34.311908 4609 log.cc:824] T 00000000000000000000000000000000 P cbbde7f6994a4afaafe4eb3ba0114975: Log is configured to *not* fsync() on all Append() calls
I20260811 09:13:34.315448 4609 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cbbde7f6994a4afaafe4eb3ba0114975: No bootstrap required, opened a new log
I20260811 09:13:34.322471 4612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:34.315455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.126:43077 (local address 127.0.0.1:34247)
0811 09:13:34.315644 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:34.315650 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:34.316100 (+ 450us) server_negotiation.cc:407] Connection header received
0811 09:13:34.316289 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:34.316294 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:34.316397 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:34.316491 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:34.317187 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:34.318264 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:34.319680 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:34.320201 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:34.320364 (+ 163us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:34.321659 (+ 1295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:13:34.321665 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:13:34.321672 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:13:34.321693 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:13:34.321737 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:13:34.321754 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:13:34.321869 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:34.322105 (+ 236us) server_negotiation.cc:299] Negotiation successful
0811 09:13:34.322164 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 09:13:34.325250 4609 raft_consensus.cc:348] T 00000000000000000000000000000000 P cbbde7f6994a4afaafe4eb3ba0114975 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbbde7f6994a4afaafe4eb3ba0114975" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34247 } }
I20260811 09:13:34.325724 4609 raft_consensus.cc:374] T 00000000000000000000000000000000 P cbbde7f6994a4afaafe4eb3ba0114975 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:13:34.325845 4609 raft_consensus.cc:729] T 00000000000000000000000000000000 P cbbde7f6994a4afaafe4eb3ba0114975 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cbbde7f6994a4afaafe4eb3ba0114975, State: Initialized, Role: FOLLOWER
I20260811 09:13:34.326524 4609 consensus_queue.cc:260] T 00000000000000000000000000000000 P cbbde7f6994a4afaafe4eb3ba0114975 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbbde7f6994a4afaafe4eb3ba0114975" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34247 } }
I20260811 09:13:34.326758 4609 raft_consensus.cc:388] T 00000000000000000000000000000000 P cbbde7f6994a4afaafe4eb3ba0114975 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:13:34.326898 4609 raft_consensus.cc:482] T 00000000000000000000000000000000 P cbbde7f6994a4afaafe4eb3ba0114975 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:13:34.327057 4609 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cbbde7f6994a4afaafe4eb3ba0114975 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:13:34.330248 4609 raft_consensus.cc:504] T 00000000000000000000000000000000 P cbbde7f6994a4afaafe4eb3ba0114975 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbbde7f6994a4afaafe4eb3ba0114975" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34247 } }
I20260811 09:13:34.330808 4609 leader_election.cc:302] T 00000000000000000000000000000000 P cbbde7f6994a4afaafe4eb3ba0114975 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cbbde7f6994a4afaafe4eb3ba0114975; no voters:
I20260811 09:13:34.331467 4609 leader_election.cc:288] T 00000000000000000000000000000000 P cbbde7f6994a4afaafe4eb3ba0114975 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:13:34.331900 4619 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cbbde7f6994a4afaafe4eb3ba0114975 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:13:34.333240 4619 raft_consensus.cc:686] T 00000000000000000000000000000000 P cbbde7f6994a4afaafe4eb3ba0114975 [term 1 LEADER]: Becoming Leader. State: Replica: cbbde7f6994a4afaafe4eb3ba0114975, State: Running, Role: LEADER
I20260811 09:13:34.333863 4609 sys_catalog.cc:564] T 00000000000000000000000000000000 P cbbde7f6994a4afaafe4eb3ba0114975 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:13:34.333988 4619 consensus_queue.cc:237] T 00000000000000000000000000000000 P cbbde7f6994a4afaafe4eb3ba0114975 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbbde7f6994a4afaafe4eb3ba0114975" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34247 } }
I20260811 09:13:34.337193 4623 sys_catalog.cc:455] T 00000000000000000000000000000000 P cbbde7f6994a4afaafe4eb3ba0114975 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cbbde7f6994a4afaafe4eb3ba0114975" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbbde7f6994a4afaafe4eb3ba0114975" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34247 } } }
I20260811 09:13:34.337342 4625 sys_catalog.cc:455] T 00000000000000000000000000000000 P cbbde7f6994a4afaafe4eb3ba0114975 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cbbde7f6994a4afaafe4eb3ba0114975. Latest consensus state: current_term: 1 leader_uuid: "cbbde7f6994a4afaafe4eb3ba0114975" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbbde7f6994a4afaafe4eb3ba0114975" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34247 } } }
I20260811 09:13:34.337653 4623 sys_catalog.cc:458] T 00000000000000000000000000000000 P cbbde7f6994a4afaafe4eb3ba0114975 [sys.catalog]: This master's current role is: LEADER
I20260811 09:13:34.337654 4625 sys_catalog.cc:458] T 00000000000000000000000000000000 P cbbde7f6994a4afaafe4eb3ba0114975 [sys.catalog]: This master's current role is: LEADER
W20260811 09:13:34.344923 4650 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cbbde7f6994a4afaafe4eb3ba0114975: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 09:13:34.345042 4650 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 09:13:34.345188 4639 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:13:34.347450 4639 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:13:34.356648 4639 catalog_manager.cc:1347] Generated new cluster ID: f0097c3340244aa9b7beb8a45c604ecd
I20260811 09:13:34.356775 4639 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:13:34.380972 4639 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:13:34.382289 4639 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:13:34.390100 4639 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cbbde7f6994a4afaafe4eb3ba0114975: Generated new TSK 0
I20260811 09:13:34.391191 4639 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 09:13:34.397250 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 4462
W20260811 09:13:34.398581 4599 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.126:42473
W20260811 09:13:34.399394 29290 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 4462: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 09:13:34.399554 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 4462
W20260811 09:13:34.423276 4443 connection.cc:535] client connection to 127.0.0.1:34247 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34247): Connection reset by peer (error 104)
2026-08-11T09:13:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (2200 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786439250192320-29290-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:13:34 dist-test-slave-xt4k krb5kdc[4661](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:13:34 dist-test-slave-xt4k krb5kdc[4661](info): set up 2 sockets
Aug 11 09:13:34 dist-test-slave-xt4k krb5kdc[4661](info): commencing operation
krb5kdc: starting...
W20260811 09:13:35.706455 29290 mini_kdc.cc:121] Time spent starting KDC: real 1.264s user 0.000s sys 0.048s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786439250192320-29290-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786439250192320-29290-0/krb5kdc/test-user.keytab.
Aug 11 09:13:35 dist-test-slave-xt4k krb5kdc[4661](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439615, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:13:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:13:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786439250192320-29290-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786439250192320-29290-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 09:13:36.041654 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786439250192320-29290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786439250192320-29290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786439250192320-29290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42025
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786439250192320-29290-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42025
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:13:36.394312 4687 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:13:36.394662 4687 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:13:36.394810 4687 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:13:36.402257 4687 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:13:36.402416 4687 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:13:36.402510 4687 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:13:36.402592 4687 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:13:36.402655 4687 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:13:36.412642 4687 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786439250192320-29290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786439250192320-29290-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42025
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786439250192320-29290-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42025
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786439250192320-29290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:13:36.413980 4687 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:13:36.415271 4687 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:13:36.426950 4698 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:13:36.427047 4693 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:13:36.427081 4695 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:13:36.427562 4687 server_base.cc:1034] running on GCE node
I20260811 09:13:36.428330 4687 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:13:36.429817 4687 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:13:36.431038 4687 hybrid_clock.cc:648] HybridClock initialized: now 1786439616430980 us; error 46 us; skew 500 ppm
I20260811 09:13:36.431391 4687 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:13:36 dist-test-slave-xt4k krb5kdc[4661](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439616, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:13:36.434592 4687 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 09:13:36.436088 4687 webserver.cc:460] Webserver started at http://127.0.0.1:45095/ using document root <none> and password file <none>
I20260811 09:13:36.436825 4687 fs_manager.cc:359] Metadata directory not provided
I20260811 09:13:36.436932 4687 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:13:36.437201 4687 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:13:36.440461 4687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786439250192320-29290-0/minicluster-data/master-0/data/instance:
uuid: "4960ef95dfb0435d8adb09c554a4bcae"
format_stamp: "Formatted at 2026-08-11 09:13:36 on dist-test-slave-xt4k"
server_key: "ad0f92cd9d2ad2f1b72b1b75bebb58f1"
server_key_iv: "76fa90fb4a51f03dd439462dec1a27e2"
server_key_version: "encryptionkey@0"
I20260811 09:13:36.441440 4687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786439250192320-29290-0/minicluster-data/master-0/wal/instance:
uuid: "4960ef95dfb0435d8adb09c554a4bcae"
format_stamp: "Formatted at 2026-08-11 09:13:36 on dist-test-slave-xt4k"
server_key: "ad0f92cd9d2ad2f1b72b1b75bebb58f1"
server_key_iv: "76fa90fb4a51f03dd439462dec1a27e2"
server_key_version: "encryptionkey@0"
I20260811 09:13:36.447088 4687 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 09:13:36.450805 4713 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:13:36.452211 4687 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 09:13:36.452368 4687 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786439250192320-29290-0/minicluster-data/master-0/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786439250192320-29290-0/minicluster-data/master-0/wal
uuid: "4960ef95dfb0435d8adb09c554a4bcae"
format_stamp: "Formatted at 2026-08-11 09:13:36 on dist-test-slave-xt4k"
server_key: "ad0f92cd9d2ad2f1b72b1b75bebb58f1"
server_key_iv: "76fa90fb4a51f03dd439462dec1a27e2"
server_key_version: "encryptionkey@0"
I20260811 09:13:36.452620 4687 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786439250192320-29290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786439250192320-29290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786439250192320-29290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:13:36.458524 4687 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:13:36.462787 4687 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:13:36.463038 4687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:13:36.481392 4687 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42025
I20260811 09:13:36.481422 4802 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42025 every 8 connection(s)
I20260811 09:13:36.482852 4687 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
I20260811 09:13:36.484275 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 4687
I20260811 09:13:36.484478 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786439250192320-29290-0/minicluster-data/master-0/wal/instance
I20260811 09:13:36.485033 29290 external_mini_cluster.cc:1428] Setting key 8725b8e7b700f8db9d01315f949172db
I20260811 09:13:36.487540 4803 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:13:36.492822 4811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:36.486952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.126:59753 (local address 127.0.0.1:42025)
0811 09:13:36.487798 (+ 846us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:36.487813 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:36.487862 (+ 49us) server_negotiation.cc:407] Connection header received
0811 09:13:36.489524 (+ 1662us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:36.489550 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:36.489876 (+ 326us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:36.490205 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:36.491202 (+ 997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:13:36.491208 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:13:36.491220 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:13:36.491237 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:13:36.491312 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:13:36.491331 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:13:36.491530 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:36.491631 (+ 101us) server_negotiation.cc:299] Negotiation successful
0811 09:13:36.491824 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":614,"thread_start_us":331,"threads_started":1}
I20260811 09:13:36.496810 29290 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 09:13:36.499966 4803 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4960ef95dfb0435d8adb09c554a4bcae: Bootstrap starting.
I20260811 09:13:36.503779 4803 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4960ef95dfb0435d8adb09c554a4bcae: Neither blocks nor log segments found. Creating new log.
I20260811 09:13:36.503970 4811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:36.501414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.126:50649 (local address 127.0.0.1:42025)
0811 09:13:36.501627 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:36.501635 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:36.502019 (+ 384us) server_negotiation.cc:407] Connection header received
0811 09:13:36.502172 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:36.502178 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:36.502245 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:36.502379 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:36.503263 (+ 884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:13:36.503269 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:13:36.503273 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:13:36.503292 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:13:36.503342 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:13:36.503363 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:13:36.503484 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:36.503541 (+ 57us) server_negotiation.cc:299] Negotiation successful
0811 09:13:36.503604 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 09:13:36.505081 4803 log.cc:824] T 00000000000000000000000000000000 P 4960ef95dfb0435d8adb09c554a4bcae: Log is configured to *not* fsync() on all Append() calls
I20260811 09:13:36.508155 4803 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4960ef95dfb0435d8adb09c554a4bcae: No bootstrap required, opened a new log
I20260811 09:13:36.515792 4803 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4960ef95dfb0435d8adb09c554a4bcae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4960ef95dfb0435d8adb09c554a4bcae" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42025 } }
I20260811 09:13:36.516194 4803 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4960ef95dfb0435d8adb09c554a4bcae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:13:36.516306 4803 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4960ef95dfb0435d8adb09c554a4bcae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4960ef95dfb0435d8adb09c554a4bcae, State: Initialized, Role: FOLLOWER
I20260811 09:13:36.516955 4803 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4960ef95dfb0435d8adb09c554a4bcae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4960ef95dfb0435d8adb09c554a4bcae" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42025 } }
I20260811 09:13:36.517174 4803 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4960ef95dfb0435d8adb09c554a4bcae [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:13:36.517309 4803 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4960ef95dfb0435d8adb09c554a4bcae [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:13:36.517467 4803 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4960ef95dfb0435d8adb09c554a4bcae [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:13:36.519773 4803 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4960ef95dfb0435d8adb09c554a4bcae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4960ef95dfb0435d8adb09c554a4bcae" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42025 } }
I20260811 09:13:36.520394 4803 leader_election.cc:302] T 00000000000000000000000000000000 P 4960ef95dfb0435d8adb09c554a4bcae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4960ef95dfb0435d8adb09c554a4bcae; no voters:
I20260811 09:13:36.521045 4803 leader_election.cc:288] T 00000000000000000000000000000000 P 4960ef95dfb0435d8adb09c554a4bcae [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:13:36.521176 4820 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4960ef95dfb0435d8adb09c554a4bcae [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:13:36.522887 4803 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4960ef95dfb0435d8adb09c554a4bcae [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:13:36.523289 4820 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4960ef95dfb0435d8adb09c554a4bcae [term 1 LEADER]: Becoming Leader. State: Replica: 4960ef95dfb0435d8adb09c554a4bcae, State: Running, Role: LEADER
I20260811 09:13:36.523809 4820 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4960ef95dfb0435d8adb09c554a4bcae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4960ef95dfb0435d8adb09c554a4bcae" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42025 } }
I20260811 09:13:36.526921 4825 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4960ef95dfb0435d8adb09c554a4bcae [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4960ef95dfb0435d8adb09c554a4bcae. Latest consensus state: current_term: 1 leader_uuid: "4960ef95dfb0435d8adb09c554a4bcae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4960ef95dfb0435d8adb09c554a4bcae" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42025 } } }
I20260811 09:13:36.526947 4824 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4960ef95dfb0435d8adb09c554a4bcae [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4960ef95dfb0435d8adb09c554a4bcae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4960ef95dfb0435d8adb09c554a4bcae" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42025 } } }
I20260811 09:13:36.527585 4825 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4960ef95dfb0435d8adb09c554a4bcae [sys.catalog]: This master's current role is: LEADER
I20260811 09:13:36.527619 4824 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4960ef95dfb0435d8adb09c554a4bcae [sys.catalog]: This master's current role is: LEADER
I20260811 09:13:36.529218 4844 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:13:36.533667 4844 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:13:36.542124 4844 catalog_manager.cc:1347] Generated new cluster ID: 70818e78dd374024a518b8be08bc8342
I20260811 09:13:36.542239 4844 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:13:36.558532 4844 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:13:36.559083 4844 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:13:36.568710 4844 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4960ef95dfb0435d8adb09c554a4bcae: Generated new TSK 0
I20260811 09:13:36.569629 4844 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 09:13:36.582841 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 4687
W20260811 09:13:36.584141 4793 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.126:59753
W20260811 09:13:36.585019 29290 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 4687: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 09:13:36.585172 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 4687
2026-08-11T09:13:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (2189 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786439250192320-29290-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:13:36 dist-test-slave-xt4k krb5kdc[4859](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:13:36 dist-test-slave-xt4k krb5kdc[4859](info): set up 2 sockets
Aug 11 09:13:36 dist-test-slave-xt4k krb5kdc[4859](info): commencing operation
krb5kdc: starting...
W20260811 09:13:37.898599 29290 mini_kdc.cc:121] Time spent starting KDC: real 1.266s user 0.005s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786439250192320-29290-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786439250192320-29290-0/krb5kdc/test-user.keytab.
Aug 11 09:13:38 dist-test-slave-xt4k krb5kdc[4859](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439618, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:13:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:13:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.154.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.154.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.154.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab.
Entry for principal kudu/127.28.154.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab.
I20260811 09:13:38.234071 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786439250192320-29290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786439250192320-29290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786439250192320-29290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.154.190:38979
--webserver_interface=127.28.154.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab
--principal=kudu/127.28.154.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:40233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.154.190:38979
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:13:38.584890 4888 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:13:38.585182 4888 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:13:38.585271 4888 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:13:38.592528 4888 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:13:38.592654 4888 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:13:38.592708 4888 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:13:38.592769 4888 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:13:38.592813 4888 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:13:38.603134 4888 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:40233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786439250192320-29290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786439250192320-29290-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.154.190:38979
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab
--principal=kudu/127.28.154.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.154.190:38979
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.154.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786439250192320-29290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:13:38.604446 4888 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:13:38.605711 4888 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:13:38.617312 4895 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:13:38.617367 4904 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:13:38.617424 4901 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:13:38.618700 4888 server_base.cc:1034] running on GCE node
I20260811 09:13:38.619462 4888 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:13:38.621004 4888 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:13:38.622221 4888 hybrid_clock.cc:648] HybridClock initialized: now 1786439618622167 us; error 46 us; skew 500 ppm
I20260811 09:13:38.622591 4888 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:13:38 dist-test-slave-xt4k krb5kdc[4859](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439618, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:13:38.625821 4888 init.cc:377] Logged in from keytab as kudu/127.28.154.190@KRBTEST.COM (short username kudu)
I20260811 09:13:38.627285 4888 webserver.cc:460] Webserver started at http://127.28.154.190:38909/ using document root <none> and password file <none>
I20260811 09:13:38.627982 4888 fs_manager.cc:359] Metadata directory not provided
I20260811 09:13:38.628108 4888 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:13:38.628433 4888 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:13:38.631424 4888 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786439250192320-29290-0/minicluster-data/master-0/data/instance:
uuid: "908bd7918fd94220bfbf72fc78b2cdeb"
format_stamp: "Formatted at 2026-08-11 09:13:38 on dist-test-slave-xt4k"
server_key: "6d0750e06e417ae9cf1e203e54a6ba21"
server_key_iv: "3f98af99b31668b3ab3355e62a1faccb"
server_key_version: "encryptionkey@0"
I20260811 09:13:38.632201 4888 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786439250192320-29290-0/minicluster-data/master-0/wal/instance:
uuid: "908bd7918fd94220bfbf72fc78b2cdeb"
format_stamp: "Formatted at 2026-08-11 09:13:38 on dist-test-slave-xt4k"
server_key: "6d0750e06e417ae9cf1e203e54a6ba21"
server_key_iv: "3f98af99b31668b3ab3355e62a1faccb"
server_key_version: "encryptionkey@0"
I20260811 09:13:38.637553 4888 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 09:13:38.641266 4918 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:13:38.642659 4888 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 09:13:38.642829 4888 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786439250192320-29290-0/minicluster-data/master-0/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786439250192320-29290-0/minicluster-data/master-0/wal
uuid: "908bd7918fd94220bfbf72fc78b2cdeb"
format_stamp: "Formatted at 2026-08-11 09:13:38 on dist-test-slave-xt4k"
server_key: "6d0750e06e417ae9cf1e203e54a6ba21"
server_key_iv: "3f98af99b31668b3ab3355e62a1faccb"
server_key_version: "encryptionkey@0"
I20260811 09:13:38.643018 4888 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786439250192320-29290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786439250192320-29290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786439250192320-29290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:13:38.670310 4888 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:13:38.674902 4888 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:13:38.675176 4888 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:13:38.693714 4888 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.190:38979
I20260811 09:13:38.693754 5003 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.190:38979 every 8 connection(s)
I20260811 09:13:38.695096 4888 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
I20260811 09:13:38.697137 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 4888
I20260811 09:13:38.697329 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786439250192320-29290-0/minicluster-data/master-0/wal/instance
I20260811 09:13:38.697880 29290 external_mini_cluster.cc:1428] Setting key 472d7aca446b50c3e5340a147e8c900b
I20260811 09:13:38.699775 5004 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 09:13:38 dist-test-slave-xt4k krb5kdc[4859](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439618, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.154.190@KRBTEST.COM
I20260811 09:13:38.713290 5004 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 908bd7918fd94220bfbf72fc78b2cdeb: Bootstrap starting.
I20260811 09:13:38.716867 5004 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 908bd7918fd94220bfbf72fc78b2cdeb: Neither blocks nor log segments found. Creating new log.
I20260811 09:13:38.718111 5004 log.cc:824] T 00000000000000000000000000000000 P 908bd7918fd94220bfbf72fc78b2cdeb: Log is configured to *not* fsync() on all Append() calls
I20260811 09:13:38.721112 5004 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 908bd7918fd94220bfbf72fc78b2cdeb: No bootstrap required, opened a new log
I20260811 09:13:38.721796 5008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:38.699737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46499 (local address 127.28.154.190:38979)
0811 09:13:38.700585 (+ 848us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:38.700601 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:38.700651 (+ 50us) server_negotiation.cc:407] Connection header received
0811 09:13:38.702770 (+ 2119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:38.702801 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:38.703334 (+ 533us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:38.703766 (+ 432us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:38.705169 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:38.707219 (+ 2050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:38.708498 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:38.709518 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:38.709709 (+ 191us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:38.714026 (+ 4317us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:13:38.714035 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:13:38.714051 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:13:38.714094 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:13:38.717340 (+ 3246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:13:38.718354 (+ 1014us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:13:38.718363 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:13:38.718369 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:13:38.718569 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:13:38.719043 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:13:38.719047 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:13:38.719049 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:13:38.719599 (+ 550us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:13:38.719916 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:38.720457 (+ 541us) server_negotiation.cc:299] Negotiation successful
0811 09:13:38.720749 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":599,"thread_start_us":322,"threads_started":1}
I20260811 09:13:38.725791 29290 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 09:13:38.729749 5004 raft_consensus.cc:348] T 00000000000000000000000000000000 P 908bd7918fd94220bfbf72fc78b2cdeb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "908bd7918fd94220bfbf72fc78b2cdeb" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 38979 } }
I20260811 09:13:38.730244 5004 raft_consensus.cc:374] T 00000000000000000000000000000000 P 908bd7918fd94220bfbf72fc78b2cdeb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:13:38.730350 5004 raft_consensus.cc:729] T 00000000000000000000000000000000 P 908bd7918fd94220bfbf72fc78b2cdeb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 908bd7918fd94220bfbf72fc78b2cdeb, State: Initialized, Role: FOLLOWER
I20260811 09:13:38.730973 5004 consensus_queue.cc:260] T 00000000000000000000000000000000 P 908bd7918fd94220bfbf72fc78b2cdeb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "908bd7918fd94220bfbf72fc78b2cdeb" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 38979 } }
I20260811 09:13:38.731249 5004 raft_consensus.cc:388] T 00000000000000000000000000000000 P 908bd7918fd94220bfbf72fc78b2cdeb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:13:38.731395 5004 raft_consensus.cc:482] T 00000000000000000000000000000000 P 908bd7918fd94220bfbf72fc78b2cdeb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:13:38.731571 5004 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 908bd7918fd94220bfbf72fc78b2cdeb [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:13:38.733870 5004 raft_consensus.cc:504] T 00000000000000000000000000000000 P 908bd7918fd94220bfbf72fc78b2cdeb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "908bd7918fd94220bfbf72fc78b2cdeb" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 38979 } }
I20260811 09:13:38.734360 5004 leader_election.cc:302] T 00000000000000000000000000000000 P 908bd7918fd94220bfbf72fc78b2cdeb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 908bd7918fd94220bfbf72fc78b2cdeb; no voters:
I20260811 09:13:38.734987 5004 leader_election.cc:288] T 00000000000000000000000000000000 P 908bd7918fd94220bfbf72fc78b2cdeb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:13:38.735118 5019 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 908bd7918fd94220bfbf72fc78b2cdeb [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:13:38.736099 5019 raft_consensus.cc:686] T 00000000000000000000000000000000 P 908bd7918fd94220bfbf72fc78b2cdeb [term 1 LEADER]: Becoming Leader. State: Replica: 908bd7918fd94220bfbf72fc78b2cdeb, State: Running, Role: LEADER
I20260811 09:13:38.736739 5019 consensus_queue.cc:237] T 00000000000000000000000000000000 P 908bd7918fd94220bfbf72fc78b2cdeb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "908bd7918fd94220bfbf72fc78b2cdeb" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 38979 } }
I20260811 09:13:38.737433 5004 sys_catalog.cc:564] T 00000000000000000000000000000000 P 908bd7918fd94220bfbf72fc78b2cdeb [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:13:38.738765 5021 sys_catalog.cc:455] T 00000000000000000000000000000000 P 908bd7918fd94220bfbf72fc78b2cdeb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 908bd7918fd94220bfbf72fc78b2cdeb. Latest consensus state: current_term: 1 leader_uuid: "908bd7918fd94220bfbf72fc78b2cdeb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "908bd7918fd94220bfbf72fc78b2cdeb" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 38979 } } }
I20260811 09:13:38.739153 5021 sys_catalog.cc:458] T 00000000000000000000000000000000 P 908bd7918fd94220bfbf72fc78b2cdeb [sys.catalog]: This master's current role is: LEADER
I20260811 09:13:38.741353 5025 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:13:38.740836 5020 sys_catalog.cc:455] T 00000000000000000000000000000000 P 908bd7918fd94220bfbf72fc78b2cdeb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "908bd7918fd94220bfbf72fc78b2cdeb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "908bd7918fd94220bfbf72fc78b2cdeb" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 38979 } } }
I20260811 09:13:38.742336 5020 sys_catalog.cc:458] T 00000000000000000000000000000000 P 908bd7918fd94220bfbf72fc78b2cdeb [sys.catalog]: This master's current role is: LEADER
I20260811 09:13:38.746644 5008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:38.729960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54773 (local address 127.28.154.190:38979)
0811 09:13:38.730103 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:38.730107 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:38.730592 (+ 485us) server_negotiation.cc:407] Connection header received
0811 09:13:38.730770 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:38.730774 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:38.730833 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:38.730910 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:38.732542 (+ 1632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:38.733318 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:38.734581 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:38.735084 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:38.735216 (+ 132us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:38.737824 (+ 2608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:13:38.737831 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:13:38.737836 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:13:38.737874 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:13:38.741772 (+ 3898us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:13:38.742803 (+ 1031us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:13:38.742809 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:13:38.742813 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:13:38.743090 (+ 277us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:13:38.743779 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:13:38.743784 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:13:38.743787 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:13:38.744205 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:13:38.745807 (+ 1602us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:38.746029 (+ 222us) server_negotiation.cc:299] Negotiation successful
0811 09:13:38.746245 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 09:13:38.753168 5025 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:13:38.763646 5025 catalog_manager.cc:1347] Generated new cluster ID: 1bf485a765584712bb23e918d4bd052b
I20260811 09:13:38.763772 5025 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:13:38.780998 5025 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:13:38.782971 5025 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:13:38.791726 5025 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 908bd7918fd94220bfbf72fc78b2cdeb: Generated new TSK 0
I20260811 09:13:38.792835 5025 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 09:13:38.806107 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 4888
I20260811 09:13:38.807277 4995 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:13:38.899930 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 4888
W20260811 09:13:38.922542 4855 connection.cc:535] client connection to 127.28.154.190:38979 recv error: Network error: failed to read from TLS socket (remote: 127.28.154.190:38979): Connection reset by peer (error 104)
2026-08-11T09:13:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (2309 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786439250192320-29290-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:13:39 dist-test-slave-xt4k krb5kdc[5051](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:13:39 dist-test-slave-xt4k krb5kdc[5051](info): set up 2 sockets
Aug 11 09:13:39 dist-test-slave-xt4k krb5kdc[5051](info): commencing operation
krb5kdc: starting...
W20260811 09:13:40.197640 29290 mini_kdc.cc:121] Time spent starting KDC: real 1.256s user 0.000s sys 0.050s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786439250192320-29290-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786439250192320-29290-0/krb5kdc/test-user.keytab.
Aug 11 09:13:40 dist-test-slave-xt4k krb5kdc[5051](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439620, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:13:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:13:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.154.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.154.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.154.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab.
Entry for principal kudu/127.28.154.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab.
I20260811 09:13:40.532699 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786439250192320-29290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786439250192320-29290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786439250192320-29290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.154.190:35729
--webserver_interface=127.28.154.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab
--principal=kudu/127.28.154.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:43271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.154.190:35729
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:13:40.880353 5079 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:13:40.880666 5079 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:13:40.880816 5079 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:13:40.888180 5079 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:13:40.888311 5079 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:13:40.888391 5079 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:13:40.888484 5079 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:13:40.888541 5079 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:13:40.898622 5079 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:43271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786439250192320-29290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786439250192320-29290-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.154.190:35729
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab
--principal=kudu/127.28.154.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.154.190:35729
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.154.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786439250192320-29290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:13:40.899981 5079 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:13:40.901345 5079 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:13:40.913686 5100 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:13:40.913879 5090 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:13:40.913839 5092 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:13:40.915110 5079 server_base.cc:1034] running on GCE node
I20260811 09:13:40.916013 5079 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:13:40.917594 5079 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:13:40.918833 5079 hybrid_clock.cc:648] HybridClock initialized: now 1786439620918768 us; error 60 us; skew 500 ppm
I20260811 09:13:40.919234 5079 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:13:40 dist-test-slave-xt4k krb5kdc[5051](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439620, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:13:40.922562 5079 init.cc:377] Logged in from keytab as kudu/127.28.154.190@KRBTEST.COM (short username kudu)
I20260811 09:13:40.924428 5079 webserver.cc:460] Webserver started at http://127.28.154.190:33221/ using document root <none> and password file <none>
I20260811 09:13:40.925210 5079 fs_manager.cc:359] Metadata directory not provided
I20260811 09:13:40.925343 5079 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:13:40.925673 5079 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:13:40.929167 5079 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786439250192320-29290-0/minicluster-data/master-0/data/instance:
uuid: "6148096fa1094db5bd2bb68ec48824cd"
format_stamp: "Formatted at 2026-08-11 09:13:40 on dist-test-slave-xt4k"
server_key: "210c03fc201ef24d99c9cf257e8e3b95"
server_key_iv: "1e5ea36138c282b9b39139960e6b3a80"
server_key_version: "encryptionkey@0"
I20260811 09:13:40.930023 5079 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786439250192320-29290-0/minicluster-data/master-0/wal/instance:
uuid: "6148096fa1094db5bd2bb68ec48824cd"
format_stamp: "Formatted at 2026-08-11 09:13:40 on dist-test-slave-xt4k"
server_key: "210c03fc201ef24d99c9cf257e8e3b95"
server_key_iv: "1e5ea36138c282b9b39139960e6b3a80"
server_key_version: "encryptionkey@0"
I20260811 09:13:40.935685 5079 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 09:13:40.939646 5116 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:13:40.941105 5079 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 09:13:40.941272 5079 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786439250192320-29290-0/minicluster-data/master-0/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786439250192320-29290-0/minicluster-data/master-0/wal
uuid: "6148096fa1094db5bd2bb68ec48824cd"
format_stamp: "Formatted at 2026-08-11 09:13:40 on dist-test-slave-xt4k"
server_key: "210c03fc201ef24d99c9cf257e8e3b95"
server_key_iv: "1e5ea36138c282b9b39139960e6b3a80"
server_key_version: "encryptionkey@0"
I20260811 09:13:40.941466 5079 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786439250192320-29290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786439250192320-29290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786439250192320-29290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:13:40.955957 5079 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:13:40.960879 5079 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:13:40.961172 5079 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:13:40.979976 5079 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.190:35729
I20260811 09:13:40.979998 5204 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.190:35729 every 8 connection(s)
I20260811 09:13:40.981400 5079 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
I20260811 09:13:40.985616 5205 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:13:40.986287 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 5079
I20260811 09:13:40.986454 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786439250192320-29290-0/minicluster-data/master-0/wal/instance
I20260811 09:13:40.986910 29290 external_mini_cluster.cc:1428] Setting key 0b2629d60a34d867b3e3e50f54a411bf
I20260811 09:13:41.000444 5205 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6148096fa1094db5bd2bb68ec48824cd: Bootstrap starting.
I20260811 09:13:41.002477 5210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:40.988749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56415 (local address 127.28.154.190:35729)
0811 09:13:40.989531 (+ 782us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:40.989543 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:40.989588 (+ 45us) server_negotiation.cc:407] Connection header received
0811 09:13:40.991812 (+ 2224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:40.991845 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:40.992308 (+ 463us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:40.992748 (+ 440us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:40.993403 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:40.994808 (+ 1405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:40.996866 (+ 2058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:40.997862 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:40.998029 (+ 167us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:40.999603 (+ 1574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:13:40.999612 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:13:40.999630 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:13:41.000485 (+ 855us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:13:41.000633 (+ 148us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:13:41.000670 (+ 37us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:13:41.000844 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:41.001158 (+ 314us) server_negotiation.cc:299] Negotiation successful
0811 09:13:41.001369 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":562,"thread_start_us":353,"threads_started":1}
I20260811 09:13:41.004247 5205 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6148096fa1094db5bd2bb68ec48824cd: Neither blocks nor log segments found. Creating new log.
I20260811 09:13:41.005461 5205 log.cc:824] T 00000000000000000000000000000000 P 6148096fa1094db5bd2bb68ec48824cd: Log is configured to *not* fsync() on all Append() calls
I20260811 09:13:41.006819 29290 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 09:13:41.008646 5205 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6148096fa1094db5bd2bb68ec48824cd: No bootstrap required, opened a new log
I20260811 09:13:41.017439 5205 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6148096fa1094db5bd2bb68ec48824cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6148096fa1094db5bd2bb68ec48824cd" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 35729 } }
I20260811 09:13:41.017871 5205 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6148096fa1094db5bd2bb68ec48824cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:13:41.018050 5205 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6148096fa1094db5bd2bb68ec48824cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6148096fa1094db5bd2bb68ec48824cd, State: Initialized, Role: FOLLOWER
I20260811 09:13:41.018318 5210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:41.011330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32847 (local address 127.28.154.190:35729)
0811 09:13:41.011513 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:41.011523 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:41.011905 (+ 382us) server_negotiation.cc:407] Connection header received
0811 09:13:41.012075 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:41.012080 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:41.012150 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:41.012245 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:41.012768 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:41.013593 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:41.015044 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:41.015776 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:41.016007 (+ 231us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:41.017489 (+ 1482us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:13:41.017494 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:13:41.017497 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:13:41.017518 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:13:41.017574 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:13:41.017592 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:13:41.017691 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:41.017965 (+ 274us) server_negotiation.cc:299] Negotiation successful
0811 09:13:41.018022 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 09:13:41.018816 5205 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6148096fa1094db5bd2bb68ec48824cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6148096fa1094db5bd2bb68ec48824cd" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 35729 } }
I20260811 09:13:41.019076 5205 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6148096fa1094db5bd2bb68ec48824cd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:13:41.019225 5205 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6148096fa1094db5bd2bb68ec48824cd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:13:41.019389 5205 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6148096fa1094db5bd2bb68ec48824cd [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:13:41.021869 5205 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6148096fa1094db5bd2bb68ec48824cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6148096fa1094db5bd2bb68ec48824cd" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 35729 } }
I20260811 09:13:41.022441 5205 leader_election.cc:302] T 00000000000000000000000000000000 P 6148096fa1094db5bd2bb68ec48824cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6148096fa1094db5bd2bb68ec48824cd; no voters:
I20260811 09:13:41.023043 5205 leader_election.cc:288] T 00000000000000000000000000000000 P 6148096fa1094db5bd2bb68ec48824cd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:13:41.023203 5228 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6148096fa1094db5bd2bb68ec48824cd [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:13:41.023873 5228 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6148096fa1094db5bd2bb68ec48824cd [term 1 LEADER]: Becoming Leader. State: Replica: 6148096fa1094db5bd2bb68ec48824cd, State: Running, Role: LEADER
I20260811 09:13:41.024607 5228 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6148096fa1094db5bd2bb68ec48824cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6148096fa1094db5bd2bb68ec48824cd" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 35729 } }
I20260811 09:13:41.025403 5205 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6148096fa1094db5bd2bb68ec48824cd [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:13:41.028031 5229 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6148096fa1094db5bd2bb68ec48824cd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6148096fa1094db5bd2bb68ec48824cd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6148096fa1094db5bd2bb68ec48824cd" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 35729 } } }
I20260811 09:13:41.028504 5229 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6148096fa1094db5bd2bb68ec48824cd [sys.catalog]: This master's current role is: LEADER
I20260811 09:13:41.028512 5230 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6148096fa1094db5bd2bb68ec48824cd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6148096fa1094db5bd2bb68ec48824cd. Latest consensus state: current_term: 1 leader_uuid: "6148096fa1094db5bd2bb68ec48824cd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6148096fa1094db5bd2bb68ec48824cd" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 35729 } } }
I20260811 09:13:41.028844 5230 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6148096fa1094db5bd2bb68ec48824cd [sys.catalog]: This master's current role is: LEADER
I20260811 09:13:41.037400 5236 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:13:41.041826 5236 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:13:41.050407 5236 catalog_manager.cc:1347] Generated new cluster ID: 28aa32e7f8aa406ca805197c06e9127a
I20260811 09:13:41.050549 5236 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:13:41.069352 5236 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:13:41.071161 5236 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:13:41.084056 5236 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6148096fa1094db5bd2bb68ec48824cd: Generated new TSK 0
I20260811 09:13:41.085080 5236 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 09:13:41.099247 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 5079
W20260811 09:13:41.100603 5197 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:56415
W20260811 09:13:41.101495 29290 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 5079: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 09:13:41.101655 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 5079
W20260811 09:13:41.127763 5047 connection.cc:535] client connection to 127.28.154.190:35729 recv error: Network error: failed to read from TLS socket (remote: 127.28.154.190:35729): Connection reset by peer (error 104)
2026-08-11T09:13:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (2205 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786439250192320-29290-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:13:41 dist-test-slave-xt4k krb5kdc[5261](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:13:41 dist-test-slave-xt4k krb5kdc[5261](info): set up 2 sockets
Aug 11 09:13:41 dist-test-slave-xt4k krb5kdc[5261](info): commencing operation
krb5kdc: starting...
W20260811 09:13:42.409662 29290 mini_kdc.cc:121] Time spent starting KDC: real 1.263s user 0.000s sys 0.049s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786439250192320-29290-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786439250192320-29290-0/krb5kdc/test-user.keytab.
Aug 11 09:13:42 dist-test-slave-xt4k krb5kdc[5261](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439622, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:13:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:13:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.154.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.154.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.154.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab.
Entry for principal kudu/127.28.154.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab.
I20260811 09:13:42.747896 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786439250192320-29290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786439250192320-29290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786439250192320-29290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.154.190:37477
--webserver_interface=127.28.154.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab
--principal=kudu/127.28.154.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:40681
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.154.190:37477
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:13:43.102715 5287 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:13:43.103024 5287 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:13:43.103147 5287 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:13:43.110909 5287 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:13:43.111092 5287 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:13:43.111167 5287 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:13:43.111235 5287 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:13:43.111280 5287 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:13:43.122226 5287 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:40681
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786439250192320-29290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786439250192320-29290-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.154.190:37477
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab
--principal=kudu/127.28.154.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.154.190:37477
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.154.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786439250192320-29290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:13:43.123543 5287 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:13:43.124873 5287 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:13:43.136997 5295 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:13:43.136970 5301 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:13:43.136925 5296 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:13:43.137938 5287 server_base.cc:1034] running on GCE node
I20260811 09:13:43.138803 5287 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:13:43.140404 5287 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:13:43.141647 5287 hybrid_clock.cc:648] HybridClock initialized: now 1786439623141599 us; error 39 us; skew 500 ppm
I20260811 09:13:43.142019 5287 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:13:43 dist-test-slave-xt4k krb5kdc[5261](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439623, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:13:43.149036 5287 init.cc:377] Logged in from keytab as kudu/127.28.154.190@KRBTEST.COM (short username kudu)
I20260811 09:13:43.150710 5287 webserver.cc:460] Webserver started at http://127.28.154.190:45873/ using document root <none> and password file <none>
I20260811 09:13:43.151492 5287 fs_manager.cc:359] Metadata directory not provided
I20260811 09:13:43.151784 5287 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:13:43.152156 5287 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:13:43.155421 5287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786439250192320-29290-0/minicluster-data/master-0/data/instance:
uuid: "2547601d40294d318238b034f0e5fef1"
format_stamp: "Formatted at 2026-08-11 09:13:43 on dist-test-slave-xt4k"
server_key: "24b9ab461363cfa90824eaf65b7f69f1"
server_key_iv: "6abb8d9a087a19d15a017e3815c9656b"
server_key_version: "encryptionkey@0"
I20260811 09:13:43.156241 5287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786439250192320-29290-0/minicluster-data/master-0/wal/instance:
uuid: "2547601d40294d318238b034f0e5fef1"
format_stamp: "Formatted at 2026-08-11 09:13:43 on dist-test-slave-xt4k"
server_key: "24b9ab461363cfa90824eaf65b7f69f1"
server_key_iv: "6abb8d9a087a19d15a017e3815c9656b"
server_key_version: "encryptionkey@0"
I20260811 09:13:43.161870 5287 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260811 09:13:43.165647 5312 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:13:43.167042 5287 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 09:13:43.167217 5287 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786439250192320-29290-0/minicluster-data/master-0/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786439250192320-29290-0/minicluster-data/master-0/wal
uuid: "2547601d40294d318238b034f0e5fef1"
format_stamp: "Formatted at 2026-08-11 09:13:43 on dist-test-slave-xt4k"
server_key: "24b9ab461363cfa90824eaf65b7f69f1"
server_key_iv: "6abb8d9a087a19d15a017e3815c9656b"
server_key_version: "encryptionkey@0"
I20260811 09:13:43.167412 5287 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786439250192320-29290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786439250192320-29290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786439250192320-29290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:13:43.173468 5287 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:13:43.177796 5287 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:13:43.178089 5287 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:13:43.196264 5287 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.190:37477
I20260811 09:13:43.196295 5399 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.190:37477 every 8 connection(s)
I20260811 09:13:43.197703 5287 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
I20260811 09:13:43.201835 5400 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:13:43.203012 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 5287
I20260811 09:13:43.203184 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786439250192320-29290-0/minicluster-data/master-0/wal/instance
I20260811 09:13:43.203755 29290 external_mini_cluster.cc:1428] Setting key 0e93816c3949e583220ec0dc715543db
I20260811 09:13:43.213414 5405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:43.206262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46709 (local address 127.28.154.190:37477)
0811 09:13:43.207438 (+ 1176us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:43.207465 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:43.207585 (+ 120us) server_negotiation.cc:407] Connection header received
0811 09:13:43.209962 (+ 2377us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:43.209989 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:43.210375 (+ 386us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:43.210797 (+ 422us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:43.211773 (+ 976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:13:43.211781 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:13:43.211795 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:13:43.211809 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:13:43.211873 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:13:43.211893 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:13:43.212114 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:43.212257 (+ 143us) server_negotiation.cc:299] Negotiation successful
0811 09:13:43.212476 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":767,"thread_start_us":441,"threads_started":1}
I20260811 09:13:43.216902 5400 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2547601d40294d318238b034f0e5fef1: Bootstrap starting.
I20260811 09:13:43.217381 29290 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 09:13:43.220161 5400 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2547601d40294d318238b034f0e5fef1: Neither blocks nor log segments found. Creating new log.
I20260811 09:13:43.221304 5400 log.cc:824] T 00000000000000000000000000000000 P 2547601d40294d318238b034f0e5fef1: Log is configured to *not* fsync() on all Append() calls
I20260811 09:13:43.224418 5405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:43.221648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34453 (local address 127.28.154.190:37477)
0811 09:13:43.221843 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:43.221851 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:43.222343 (+ 492us) server_negotiation.cc:407] Connection header received
0811 09:13:43.222492 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:43.222498 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:43.222563 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:43.222679 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:43.223684 (+ 1005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:13:43.223689 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:13:43.223692 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:13:43.223709 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:13:43.223752 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:13:43.223768 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:13:43.223873 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:43.224076 (+ 203us) server_negotiation.cc:299] Negotiation successful
0811 09:13:43.224131 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 09:13:43.224962 5400 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2547601d40294d318238b034f0e5fef1: No bootstrap required, opened a new log
I20260811 09:13:43.232676 5400 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2547601d40294d318238b034f0e5fef1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2547601d40294d318238b034f0e5fef1" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 37477 } }
I20260811 09:13:43.233083 5400 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2547601d40294d318238b034f0e5fef1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:13:43.233184 5400 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2547601d40294d318238b034f0e5fef1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2547601d40294d318238b034f0e5fef1, State: Initialized, Role: FOLLOWER
I20260811 09:13:43.233798 5400 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2547601d40294d318238b034f0e5fef1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2547601d40294d318238b034f0e5fef1" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 37477 } }
I20260811 09:13:43.234036 5400 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2547601d40294d318238b034f0e5fef1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:13:43.234172 5400 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2547601d40294d318238b034f0e5fef1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:13:43.234328 5400 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2547601d40294d318238b034f0e5fef1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:13:43.236848 5400 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2547601d40294d318238b034f0e5fef1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2547601d40294d318238b034f0e5fef1" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 37477 } }
I20260811 09:13:43.237361 5400 leader_election.cc:302] T 00000000000000000000000000000000 P 2547601d40294d318238b034f0e5fef1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2547601d40294d318238b034f0e5fef1; no voters:
I20260811 09:13:43.238008 5400 leader_election.cc:288] T 00000000000000000000000000000000 P 2547601d40294d318238b034f0e5fef1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:13:43.238169 5421 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2547601d40294d318238b034f0e5fef1 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:13:43.238781 5421 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2547601d40294d318238b034f0e5fef1 [term 1 LEADER]: Becoming Leader. State: Replica: 2547601d40294d318238b034f0e5fef1, State: Running, Role: LEADER
I20260811 09:13:43.239355 5421 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2547601d40294d318238b034f0e5fef1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2547601d40294d318238b034f0e5fef1" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 37477 } }
I20260811 09:13:43.240051 5400 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2547601d40294d318238b034f0e5fef1 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:13:43.243549 5423 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2547601d40294d318238b034f0e5fef1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2547601d40294d318238b034f0e5fef1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2547601d40294d318238b034f0e5fef1" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 37477 } } }
I20260811 09:13:43.244026 5423 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2547601d40294d318238b034f0e5fef1 [sys.catalog]: This master's current role is: LEADER
I20260811 09:13:43.244540 5425 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2547601d40294d318238b034f0e5fef1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2547601d40294d318238b034f0e5fef1. Latest consensus state: current_term: 1 leader_uuid: "2547601d40294d318238b034f0e5fef1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2547601d40294d318238b034f0e5fef1" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 37477 } } }
I20260811 09:13:43.244962 5425 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2547601d40294d318238b034f0e5fef1 [sys.catalog]: This master's current role is: LEADER
I20260811 09:13:43.246567 5438 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:13:43.252566 5438 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:13:43.263907 5438 catalog_manager.cc:1347] Generated new cluster ID: 2cac60ab956d4575bf9704f1e478e029
I20260811 09:13:43.264030 5438 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:13:43.296391 5438 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:13:43.296849 5438 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:13:43.303031 5438 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2547601d40294d318238b034f0e5fef1: Generated new TSK 0
I20260811 09:13:43.303925 5438 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 09:13:43.308570 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 5287
W20260811 09:13:43.310067 5394 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:46709
W20260811 09:13:43.310787 29290 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 5287: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 09:13:43.310946 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 5287
2026-08-11T09:13:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (2208 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786439250192320-29290-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:13:43 dist-test-slave-xt4k krb5kdc[5456](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:13:43 dist-test-slave-xt4k krb5kdc[5456](info): set up 2 sockets
Aug 11 09:13:43 dist-test-slave-xt4k krb5kdc[5456](info): commencing operation
krb5kdc: starting...
W20260811 09:13:44.633298 29290 mini_kdc.cc:121] Time spent starting KDC: real 1.278s user 0.000s sys 0.054s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786439250192320-29290-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786439250192320-29290-0/krb5kdc/test-user.keytab.
Aug 11 09:13:44 dist-test-slave-xt4k krb5kdc[5456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439624, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:13:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:13:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.154.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.154.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.154.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab.
Entry for principal kudu/127.28.154.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab.
I20260811 09:13:44.953910 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786439250192320-29290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786439250192320-29290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786439250192320-29290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.154.190:33705
--webserver_interface=127.28.154.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab
--principal=kudu/127.28.154.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:36625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.154.190:33705
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:13:45.303601 5481 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:13:45.303900 5481 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:13:45.304004 5481 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:13:45.311277 5481 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:13:45.311396 5481 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:13:45.311450 5481 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:13:45.311510 5481 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:13:45.311554 5481 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:13:45.321497 5481 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:36625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786439250192320-29290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786439250192320-29290-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.154.190:33705
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab
--principal=kudu/127.28.154.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.154.190:33705
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.154.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786439250192320-29290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:13:45.322844 5481 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:13:45.324086 5481 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:13:45.335734 5493 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:13:45.335708 5499 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:13:45.335759 5496 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:13:45.336791 5481 server_base.cc:1034] running on GCE node
I20260811 09:13:45.337582 5481 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:13:45.339002 5481 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:13:45.340224 5481 hybrid_clock.cc:648] HybridClock initialized: now 1786439625340174 us; error 37 us; skew 500 ppm
I20260811 09:13:45.340608 5481 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:13:45 dist-test-slave-xt4k krb5kdc[5456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439625, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:13:45.343757 5481 init.cc:377] Logged in from keytab as kudu/127.28.154.190@KRBTEST.COM (short username kudu)
I20260811 09:13:45.345316 5481 webserver.cc:460] Webserver started at http://127.28.154.190:43479/ using document root <none> and password file <none>
I20260811 09:13:45.346027 5481 fs_manager.cc:359] Metadata directory not provided
I20260811 09:13:45.346148 5481 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:13:45.346438 5481 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:13:45.349454 5481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786439250192320-29290-0/minicluster-data/master-0/data/instance:
uuid: "ed2ad37d4d46402cb71a32a393d589f2"
format_stamp: "Formatted at 2026-08-11 09:13:45 on dist-test-slave-xt4k"
server_key: "83939ba5074e02404aff979806f757bb"
server_key_iv: "9bb9460a3e13c20f79fd3223a5d53a4b"
server_key_version: "encryptionkey@0"
I20260811 09:13:45.350193 5481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786439250192320-29290-0/minicluster-data/master-0/wal/instance:
uuid: "ed2ad37d4d46402cb71a32a393d589f2"
format_stamp: "Formatted at 2026-08-11 09:13:45 on dist-test-slave-xt4k"
server_key: "83939ba5074e02404aff979806f757bb"
server_key_iv: "9bb9460a3e13c20f79fd3223a5d53a4b"
server_key_version: "encryptionkey@0"
I20260811 09:13:45.355286 5481 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 09:13:45.358884 5510 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:13:45.360193 5481 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 09:13:45.360399 5481 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786439250192320-29290-0/minicluster-data/master-0/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786439250192320-29290-0/minicluster-data/master-0/wal
uuid: "ed2ad37d4d46402cb71a32a393d589f2"
format_stamp: "Formatted at 2026-08-11 09:13:45 on dist-test-slave-xt4k"
server_key: "83939ba5074e02404aff979806f757bb"
server_key_iv: "9bb9460a3e13c20f79fd3223a5d53a4b"
server_key_version: "encryptionkey@0"
I20260811 09:13:45.360603 5481 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786439250192320-29290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786439250192320-29290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786439250192320-29290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:13:45.378041 5481 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:13:45.382300 5481 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:13:45.382591 5481 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:13:45.400749 5481 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.190:33705
I20260811 09:13:45.400760 5603 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.190:33705 every 8 connection(s)
I20260811 09:13:45.402107 5481 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
I20260811 09:13:45.405880 5604 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:13:45.406896 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 5481
I20260811 09:13:45.407063 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786439250192320-29290-0/minicluster-data/master-0/wal/instance
I20260811 09:13:45.407506 29290 external_mini_cluster.cc:1428] Setting key a9b9b18f2d64286a60d5bdb22cdd7d91
I20260811 09:13:45.419904 5604 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed2ad37d4d46402cb71a32a393d589f2: Bootstrap starting.
Aug 11 09:13:45 dist-test-slave-xt4k krb5kdc[5456](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439624, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.154.190@KRBTEST.COM
I20260811 09:13:45.423135 5604 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ed2ad37d4d46402cb71a32a393d589f2: Neither blocks nor log segments found. Creating new log.
I20260811 09:13:45.424474 5604 log.cc:824] T 00000000000000000000000000000000 P ed2ad37d4d46402cb71a32a393d589f2: Log is configured to *not* fsync() on all Append() calls
I20260811 09:13:45.427894 5604 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed2ad37d4d46402cb71a32a393d589f2: No bootstrap required, opened a new log
I20260811 09:13:45.431425 5609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:45.409387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47111 (local address 127.28.154.190:33705)
0811 09:13:45.410291 (+ 904us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:45.410307 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:45.410369 (+ 62us) server_negotiation.cc:407] Connection header received
0811 09:13:45.412360 (+ 1991us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:45.412395 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:45.412918 (+ 523us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:45.413352 (+ 434us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:45.414664 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:45.416620 (+ 1956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:45.417993 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:45.419029 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:45.419210 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:45.423471 (+ 4261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:13:45.423482 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:13:45.423506 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:13:45.423557 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:13:45.427393 (+ 3836us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:13:45.428110 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:13:45.428117 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:13:45.428123 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:13:45.428293 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:13:45.428708 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:13:45.428714 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:13:45.428716 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:13:45.429260 (+ 544us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:13:45.429617 (+ 357us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:45.430039 (+ 422us) server_negotiation.cc:299] Negotiation successful
0811 09:13:45.430457 (+ 418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":654,"thread_start_us":370,"threads_started":1}
I20260811 09:13:45.435393 29290 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 09:13:45.436372 5604 raft_consensus.cc:348] T 00000000000000000000000000000000 P ed2ad37d4d46402cb71a32a393d589f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed2ad37d4d46402cb71a32a393d589f2" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 33705 } }
I20260811 09:13:45.436730 5604 raft_consensus.cc:374] T 00000000000000000000000000000000 P ed2ad37d4d46402cb71a32a393d589f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:13:45.436815 5604 raft_consensus.cc:729] T 00000000000000000000000000000000 P ed2ad37d4d46402cb71a32a393d589f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed2ad37d4d46402cb71a32a393d589f2, State: Initialized, Role: FOLLOWER
I20260811 09:13:45.437367 5604 consensus_queue.cc:260] T 00000000000000000000000000000000 P ed2ad37d4d46402cb71a32a393d589f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed2ad37d4d46402cb71a32a393d589f2" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 33705 } }
I20260811 09:13:45.437592 5604 raft_consensus.cc:388] T 00000000000000000000000000000000 P ed2ad37d4d46402cb71a32a393d589f2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:13:45.437714 5604 raft_consensus.cc:482] T 00000000000000000000000000000000 P ed2ad37d4d46402cb71a32a393d589f2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:13:45.437865 5604 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ed2ad37d4d46402cb71a32a393d589f2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:13:45.440099 5604 raft_consensus.cc:504] T 00000000000000000000000000000000 P ed2ad37d4d46402cb71a32a393d589f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed2ad37d4d46402cb71a32a393d589f2" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 33705 } }
I20260811 09:13:45.440763 5604 leader_election.cc:302] T 00000000000000000000000000000000 P ed2ad37d4d46402cb71a32a393d589f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ed2ad37d4d46402cb71a32a393d589f2; no voters:
I20260811 09:13:45.441411 5604 leader_election.cc:288] T 00000000000000000000000000000000 P ed2ad37d4d46402cb71a32a393d589f2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:13:45.441571 5616 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ed2ad37d4d46402cb71a32a393d589f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:13:45.442323 5616 raft_consensus.cc:686] T 00000000000000000000000000000000 P ed2ad37d4d46402cb71a32a393d589f2 [term 1 LEADER]: Becoming Leader. State: Replica: ed2ad37d4d46402cb71a32a393d589f2, State: Running, Role: LEADER
I20260811 09:13:45.443084 5616 consensus_queue.cc:237] T 00000000000000000000000000000000 P ed2ad37d4d46402cb71a32a393d589f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed2ad37d4d46402cb71a32a393d589f2" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 33705 } }
I20260811 09:13:45.446063 5604 sys_catalog.cc:564] T 00000000000000000000000000000000 P ed2ad37d4d46402cb71a32a393d589f2 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:13:45.447635 5620 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed2ad37d4d46402cb71a32a393d589f2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ed2ad37d4d46402cb71a32a393d589f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed2ad37d4d46402cb71a32a393d589f2" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 33705 } } }
I20260811 09:13:45.448086 5620 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed2ad37d4d46402cb71a32a393d589f2 [sys.catalog]: This master's current role is: LEADER
I20260811 09:13:45.448730 5619 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed2ad37d4d46402cb71a32a393d589f2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ed2ad37d4d46402cb71a32a393d589f2. Latest consensus state: current_term: 1 leader_uuid: "ed2ad37d4d46402cb71a32a393d589f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed2ad37d4d46402cb71a32a393d589f2" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 33705 } } }
I20260811 09:13:45.449177 5619 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed2ad37d4d46402cb71a32a393d589f2 [sys.catalog]: This master's current role is: LEADER
I20260811 09:13:45.453491 5625 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:13:45.459540 5625 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:13:45.469825 5609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:45.446781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43627 (local address 127.28.154.190:33705)
0811 09:13:45.446956 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:45.446961 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:45.449939 (+ 2978us) server_negotiation.cc:407] Connection header received
0811 09:13:45.450137 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:45.450143 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:45.450246 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:45.450345 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:45.452568 (+ 2223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:45.453899 (+ 1331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:45.455738 (+ 1839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:45.457220 (+ 1482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:45.457415 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:45.461154 (+ 3739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:13:45.461163 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:13:45.461169 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:13:45.461223 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:13:45.465666 (+ 4443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:13:45.466637 (+ 971us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:13:45.466643 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:13:45.466646 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:13:45.466798 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:13:45.467493 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:13:45.467499 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:13:45.467502 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:13:45.468099 (+ 597us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:13:45.468361 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:45.469039 (+ 678us) server_negotiation.cc:299] Negotiation successful
0811 09:13:45.469318 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 09:13:45.476583 5625 catalog_manager.cc:1347] Generated new cluster ID: f1bfef264a0c4c88b6859240ef264f38
I20260811 09:13:45.476722 5625 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:13:45.488821 5625 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:13:45.490826 5625 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:13:45.503772 5625 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ed2ad37d4d46402cb71a32a393d589f2: Generated new TSK 0
I20260811 09:13:45.504943 5625 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 09:13:45.516012 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 5481
I20260811 09:13:45.517401 5597 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:13:45.609257 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 5481
2026-08-11T09:13:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (2298 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786439250192320-29290-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:13:45 dist-test-slave-xt4k krb5kdc[5658](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:13:45 dist-test-slave-xt4k krb5kdc[5658](info): set up 2 sockets
Aug 11 09:13:45 dist-test-slave-xt4k krb5kdc[5658](info): commencing operation
krb5kdc: starting...
W20260811 09:13:46.901661 29290 mini_kdc.cc:121] Time spent starting KDC: real 1.248s user 0.003s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786439250192320-29290-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786439250192320-29290-0/krb5kdc/test-user.keytab.
Aug 11 09:13:47 dist-test-slave-xt4k krb5kdc[5658](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439627, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:13:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:13:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.154.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.154.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.154.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab.
Entry for principal kudu/127.28.154.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab.
I20260811 09:13:47.244148 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786439250192320-29290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786439250192320-29290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786439250192320-29290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.154.190:34613
--webserver_interface=127.28.154.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab
--principal=kudu/127.28.154.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:41927
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.154.190:34613
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:13:47.596063 5675 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:13:47.596426 5675 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:13:47.596586 5675 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:13:47.603863 5675 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:13:47.604002 5675 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:13:47.604051 5675 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:13:47.604115 5675 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:13:47.604158 5675 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:13:47.614398 5675 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:41927
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786439250192320-29290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786439250192320-29290-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.154.190:34613
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab
--principal=kudu/127.28.154.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.154.190:34613
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.154.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786439250192320-29290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:13:47.615731 5675 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:13:47.617184 5675 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:13:47.629067 5684 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:13:47.629103 5687 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:13:47.629133 5690 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:13:47.630247 5675 server_base.cc:1034] running on GCE node
I20260811 09:13:47.631098 5675 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:13:47.632514 5675 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:13:47.633735 5675 hybrid_clock.cc:648] HybridClock initialized: now 1786439627633690 us; error 32 us; skew 500 ppm
I20260811 09:13:47.634068 5675 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:13:47 dist-test-slave-xt4k krb5kdc[5658](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439627, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:13:47.637296 5675 init.cc:377] Logged in from keytab as kudu/127.28.154.190@KRBTEST.COM (short username kudu)
I20260811 09:13:47.638770 5675 webserver.cc:460] Webserver started at http://127.28.154.190:36725/ using document root <none> and password file <none>
I20260811 09:13:47.639426 5675 fs_manager.cc:359] Metadata directory not provided
I20260811 09:13:47.639541 5675 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:13:47.639807 5675 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:13:47.642824 5675 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786439250192320-29290-0/minicluster-data/master-0/data/instance:
uuid: "79499ea70a3a4c5e9c7f70a71221eb7a"
format_stamp: "Formatted at 2026-08-11 09:13:47 on dist-test-slave-xt4k"
server_key: "adad1925513c2dcbf512075afc344c39"
server_key_iv: "9bb7053317e3f2154474daec969dfa00"
server_key_version: "encryptionkey@0"
I20260811 09:13:47.643536 5675 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786439250192320-29290-0/minicluster-data/master-0/wal/instance:
uuid: "79499ea70a3a4c5e9c7f70a71221eb7a"
format_stamp: "Formatted at 2026-08-11 09:13:47 on dist-test-slave-xt4k"
server_key: "adad1925513c2dcbf512075afc344c39"
server_key_iv: "9bb7053317e3f2154474daec969dfa00"
server_key_version: "encryptionkey@0"
I20260811 09:13:47.648778 5675 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260811 09:13:47.652321 5707 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:13:47.653769 5675 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 09:13:47.653920 5675 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786439250192320-29290-0/minicluster-data/master-0/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786439250192320-29290-0/minicluster-data/master-0/wal
uuid: "79499ea70a3a4c5e9c7f70a71221eb7a"
format_stamp: "Formatted at 2026-08-11 09:13:47 on dist-test-slave-xt4k"
server_key: "adad1925513c2dcbf512075afc344c39"
server_key_iv: "9bb7053317e3f2154474daec969dfa00"
server_key_version: "encryptionkey@0"
I20260811 09:13:47.654119 5675 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786439250192320-29290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786439250192320-29290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786439250192320-29290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:13:47.670027 5675 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:13:47.674983 5675 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:13:47.675271 5675 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:13:47.692417 5675 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.190:34613
I20260811 09:13:47.692437 5801 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.190:34613 every 8 connection(s)
I20260811 09:13:47.693877 5675 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
I20260811 09:13:47.697876 5802 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:13:47.698589 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 5675
I20260811 09:13:47.698765 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786439250192320-29290-0/minicluster-data/master-0/wal/instance
I20260811 09:13:47.699200 29290 external_mini_cluster.cc:1428] Setting key 8787330f7b1607e1df382d70d61e6613
I20260811 09:13:47.712559 5802 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79499ea70a3a4c5e9c7f70a71221eb7a: Bootstrap starting.
I20260811 09:13:47.714748 5805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:47.700970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35183 (local address 127.28.154.190:34613)
0811 09:13:47.701807 (+ 837us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:47.701823 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:47.701874 (+ 51us) server_negotiation.cc:407] Connection header received
0811 09:13:47.704401 (+ 2527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:47.704436 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:47.705004 (+ 568us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:47.705501 (+ 497us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:47.706125 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:47.708001 (+ 1876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:47.709436 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:47.710531 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:47.710711 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:47.711818 (+ 1107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:13:47.711828 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:13:47.711847 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:13:47.712610 (+ 763us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:13:47.712712 (+ 102us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:13:47.712746 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:13:47.712858 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:47.713143 (+ 285us) server_negotiation.cc:299] Negotiation successful
0811 09:13:47.713423 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":584,"thread_start_us":342,"threads_started":1}
I20260811 09:13:47.716594 5802 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 79499ea70a3a4c5e9c7f70a71221eb7a: Neither blocks nor log segments found. Creating new log.
I20260811 09:13:47.717739 5802 log.cc:824] T 00000000000000000000000000000000 P 79499ea70a3a4c5e9c7f70a71221eb7a: Log is configured to *not* fsync() on all Append() calls
I20260811 09:13:47.718662 29290 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 09:13:47.720654 5802 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79499ea70a3a4c5e9c7f70a71221eb7a: No bootstrap required, opened a new log
I20260811 09:13:47.729103 5805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:47.723146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43481 (local address 127.28.154.190:34613)
0811 09:13:47.723332 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:47.723339 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:47.723720 (+ 381us) server_negotiation.cc:407] Connection header received
0811 09:13:47.723893 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:47.723896 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:47.723959 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:47.724057 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:47.724663 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:47.725415 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:47.726720 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:47.727218 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:47.727359 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:47.728413 (+ 1054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:13:47.728419 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:13:47.728422 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:13:47.728441 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:13:47.728480 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:13:47.728496 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:13:47.728573 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:47.728709 (+ 136us) server_negotiation.cc:299] Negotiation successful
0811 09:13:47.728803 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 09:13:47.728786 5802 raft_consensus.cc:348] T 00000000000000000000000000000000 P 79499ea70a3a4c5e9c7f70a71221eb7a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79499ea70a3a4c5e9c7f70a71221eb7a" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 34613 } }
I20260811 09:13:47.729275 5802 raft_consensus.cc:374] T 00000000000000000000000000000000 P 79499ea70a3a4c5e9c7f70a71221eb7a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:13:47.729398 5802 raft_consensus.cc:729] T 00000000000000000000000000000000 P 79499ea70a3a4c5e9c7f70a71221eb7a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79499ea70a3a4c5e9c7f70a71221eb7a, State: Initialized, Role: FOLLOWER
I20260811 09:13:47.730125 5802 consensus_queue.cc:260] T 00000000000000000000000000000000 P 79499ea70a3a4c5e9c7f70a71221eb7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79499ea70a3a4c5e9c7f70a71221eb7a" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 34613 } }
I20260811 09:13:47.730424 5802 raft_consensus.cc:388] T 00000000000000000000000000000000 P 79499ea70a3a4c5e9c7f70a71221eb7a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:13:47.730576 5802 raft_consensus.cc:482] T 00000000000000000000000000000000 P 79499ea70a3a4c5e9c7f70a71221eb7a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:13:47.730772 5802 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 79499ea70a3a4c5e9c7f70a71221eb7a [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:13:47.733156 5802 raft_consensus.cc:504] T 00000000000000000000000000000000 P 79499ea70a3a4c5e9c7f70a71221eb7a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79499ea70a3a4c5e9c7f70a71221eb7a" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 34613 } }
I20260811 09:13:47.733745 5802 leader_election.cc:302] T 00000000000000000000000000000000 P 79499ea70a3a4c5e9c7f70a71221eb7a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 79499ea70a3a4c5e9c7f70a71221eb7a; no voters:
I20260811 09:13:47.734365 5802 leader_election.cc:288] T 00000000000000000000000000000000 P 79499ea70a3a4c5e9c7f70a71221eb7a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:13:47.734510 5818 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 79499ea70a3a4c5e9c7f70a71221eb7a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:13:47.735162 5818 raft_consensus.cc:686] T 00000000000000000000000000000000 P 79499ea70a3a4c5e9c7f70a71221eb7a [term 1 LEADER]: Becoming Leader. State: Replica: 79499ea70a3a4c5e9c7f70a71221eb7a, State: Running, Role: LEADER
I20260811 09:13:47.735739 5818 consensus_queue.cc:237] T 00000000000000000000000000000000 P 79499ea70a3a4c5e9c7f70a71221eb7a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79499ea70a3a4c5e9c7f70a71221eb7a" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 34613 } }
I20260811 09:13:47.736445 5802 sys_catalog.cc:564] T 00000000000000000000000000000000 P 79499ea70a3a4c5e9c7f70a71221eb7a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:13:47.739176 5820 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79499ea70a3a4c5e9c7f70a71221eb7a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 79499ea70a3a4c5e9c7f70a71221eb7a. Latest consensus state: current_term: 1 leader_uuid: "79499ea70a3a4c5e9c7f70a71221eb7a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79499ea70a3a4c5e9c7f70a71221eb7a" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 34613 } } }
I20260811 09:13:47.739542 5820 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79499ea70a3a4c5e9c7f70a71221eb7a [sys.catalog]: This master's current role is: LEADER
I20260811 09:13:47.738626 5819 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79499ea70a3a4c5e9c7f70a71221eb7a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "79499ea70a3a4c5e9c7f70a71221eb7a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79499ea70a3a4c5e9c7f70a71221eb7a" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 34613 } } }
I20260811 09:13:47.745137 5819 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79499ea70a3a4c5e9c7f70a71221eb7a [sys.catalog]: This master's current role is: LEADER
I20260811 09:13:47.747964 5833 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:13:47.753541 5833 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:13:47.762677 5833 catalog_manager.cc:1347] Generated new cluster ID: feb87997a7574ea48c62a05844bc6e7f
I20260811 09:13:47.762795 5833 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:13:47.775722 5833 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:13:47.777176 5833 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:13:47.794998 5833 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 79499ea70a3a4c5e9c7f70a71221eb7a: Generated new TSK 0
I20260811 09:13:47.795964 5833 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 09:13:47.808063 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 5675
W20260811 09:13:47.809223 5795 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:35183
W20260811 09:13:47.809948 29290 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 5675: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 09:13:47.810076 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 5675
2026-08-11T09:13:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (2201 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786439250192320-29290-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:13:47 dist-test-slave-xt4k krb5kdc[5854](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:13:47 dist-test-slave-xt4k krb5kdc[5854](info): set up 2 sockets
Aug 11 09:13:47 dist-test-slave-xt4k krb5kdc[5854](info): commencing operation
krb5kdc: starting...
W20260811 09:13:49.128723 29290 mini_kdc.cc:121] Time spent starting KDC: real 1.273s user 0.000s sys 0.050s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786439250192320-29290-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786439250192320-29290-0/krb5kdc/test-user.keytab.
Aug 11 09:13:49 dist-test-slave-xt4k krb5kdc[5854](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439629, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:13:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:13:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.154.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.154.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.154.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab.
Entry for principal kudu/127.28.154.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab.
I20260811 09:13:49.467615 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786439250192320-29290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786439250192320-29290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786439250192320-29290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.154.190:33293
--webserver_interface=127.28.154.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab
--principal=kudu/127.28.154.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:35699
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.154.190:33293
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:13:49.813601 5886 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:13:49.813907 5886 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:13:49.814046 5886 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:13:49.821339 5886 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:13:49.821447 5886 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:13:49.821518 5886 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:13:49.821609 5886 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:13:49.821664 5886 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:13:49.831609 5886 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:35699
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786439250192320-29290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786439250192320-29290-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.154.190:33293
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab
--principal=kudu/127.28.154.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.154.190:33293
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.154.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786439250192320-29290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:13:49.832919 5886 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:13:49.834291 5886 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:13:49.846873 5898 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:13:49.846977 5900 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:13:49.847033 5896 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:13:49.846908 5886 server_base.cc:1034] running on GCE node
I20260811 09:13:49.848150 5886 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:13:49.849642 5886 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:13:49.850871 5886 hybrid_clock.cc:648] HybridClock initialized: now 1786439629850820 us; error 38 us; skew 500 ppm
I20260811 09:13:49.851253 5886 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:13:49 dist-test-slave-xt4k krb5kdc[5854](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439629, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:13:49.859359 5886 init.cc:377] Logged in from keytab as kudu/127.28.154.190@KRBTEST.COM (short username kudu)
I20260811 09:13:49.861058 5886 webserver.cc:460] Webserver started at http://127.28.154.190:39951/ using document root <none> and password file <none>
I20260811 09:13:49.861752 5886 fs_manager.cc:359] Metadata directory not provided
I20260811 09:13:49.861857 5886 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:13:49.862116 5886 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:13:49.865387 5886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786439250192320-29290-0/minicluster-data/master-0/data/instance:
uuid: "7ea76ecc11bd4a90920b73f82d882d62"
format_stamp: "Formatted at 2026-08-11 09:13:49 on dist-test-slave-xt4k"
server_key: "fe9a70ab7bdb6edb2e0ee12c6eff54d7"
server_key_iv: "385e42b187a8811178eab164719d7f88"
server_key_version: "encryptionkey@0"
I20260811 09:13:49.866150 5886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786439250192320-29290-0/minicluster-data/master-0/wal/instance:
uuid: "7ea76ecc11bd4a90920b73f82d882d62"
format_stamp: "Formatted at 2026-08-11 09:13:49 on dist-test-slave-xt4k"
server_key: "fe9a70ab7bdb6edb2e0ee12c6eff54d7"
server_key_iv: "385e42b187a8811178eab164719d7f88"
server_key_version: "encryptionkey@0"
I20260811 09:13:49.871466 5886 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 09:13:49.875046 5909 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:13:49.876456 5886 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 09:13:49.876626 5886 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786439250192320-29290-0/minicluster-data/master-0/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786439250192320-29290-0/minicluster-data/master-0/wal
uuid: "7ea76ecc11bd4a90920b73f82d882d62"
format_stamp: "Formatted at 2026-08-11 09:13:49 on dist-test-slave-xt4k"
server_key: "fe9a70ab7bdb6edb2e0ee12c6eff54d7"
server_key_iv: "385e42b187a8811178eab164719d7f88"
server_key_version: "encryptionkey@0"
I20260811 09:13:49.876813 5886 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786439250192320-29290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786439250192320-29290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786439250192320-29290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:13:49.882637 5886 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:13:49.887075 5886 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:13:49.887374 5886 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:13:49.905926 5886 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.190:33293
I20260811 09:13:49.905963 6001 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.190:33293 every 8 connection(s)
I20260811 09:13:49.907310 5886 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
I20260811 09:13:49.910578 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 5886
I20260811 09:13:49.910763 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786439250192320-29290-0/minicluster-data/master-0/wal/instance
I20260811 09:13:49.911314 29290 external_mini_cluster.cc:1428] Setting key d4b05a8151f144f10424cb0644d57efd
I20260811 09:13:49.911676 6002 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:13:49.919709 6008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:49.913239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42765 (local address 127.28.154.190:33293)
0811 09:13:49.914015 (+ 776us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:49.914027 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:49.914067 (+ 40us) server_negotiation.cc:407] Connection header received
0811 09:13:49.916421 (+ 2354us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:49.916446 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:49.916819 (+ 373us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:49.917209 (+ 390us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:49.918175 (+ 966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:13:49.918181 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:13:49.918206 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:13:49.918221 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:13:49.918284 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:13:49.918306 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:13:49.918530 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:49.918672 (+ 142us) server_negotiation.cc:299] Negotiation successful
0811 09:13:49.918849 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":562,"thread_start_us":348,"threads_started":1}
I20260811 09:13:49.923364 29290 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 09:13:49.924769 6002 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ea76ecc11bd4a90920b73f82d882d62: Bootstrap starting.
I20260811 09:13:49.928145 6002 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7ea76ecc11bd4a90920b73f82d882d62: Neither blocks nor log segments found. Creating new log.
I20260811 09:13:49.929373 6002 log.cc:824] T 00000000000000000000000000000000 P 7ea76ecc11bd4a90920b73f82d882d62: Log is configured to *not* fsync() on all Append() calls
I20260811 09:13:49.930155 6008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:49.927651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33829 (local address 127.28.154.190:33293)
0811 09:13:49.927815 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:49.927821 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:49.928225 (+ 404us) server_negotiation.cc:407] Connection header received
0811 09:13:49.928407 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:49.928412 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:49.928467 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:49.928566 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:49.929473 (+ 907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:13:49.929477 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:13:49.929480 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 09:13:49.929503 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:13:49.929550 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 09:13:49.929571 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:13:49.929669 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:49.929793 (+ 124us) server_negotiation.cc:299] Negotiation successful
0811 09:13:49.929858 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 09:13:49.932641 6002 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ea76ecc11bd4a90920b73f82d882d62: No bootstrap required, opened a new log
I20260811 09:13:49.940675 6002 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7ea76ecc11bd4a90920b73f82d882d62 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ea76ecc11bd4a90920b73f82d882d62" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 33293 } }
I20260811 09:13:49.941090 6002 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7ea76ecc11bd4a90920b73f82d882d62 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 09:13:49.941198 6002 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7ea76ecc11bd4a90920b73f82d882d62 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ea76ecc11bd4a90920b73f82d882d62, State: Initialized, Role: FOLLOWER
I20260811 09:13:49.941807 6002 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7ea76ecc11bd4a90920b73f82d882d62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ea76ecc11bd4a90920b73f82d882d62" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 33293 } }
I20260811 09:13:49.942044 6002 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7ea76ecc11bd4a90920b73f82d882d62 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 09:13:49.942176 6002 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7ea76ecc11bd4a90920b73f82d882d62 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 09:13:49.942332 6002 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7ea76ecc11bd4a90920b73f82d882d62 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:13:49.944741 6002 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7ea76ecc11bd4a90920b73f82d882d62 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ea76ecc11bd4a90920b73f82d882d62" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 33293 } }
I20260811 09:13:49.945350 6002 leader_election.cc:302] T 00000000000000000000000000000000 P 7ea76ecc11bd4a90920b73f82d882d62 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7ea76ecc11bd4a90920b73f82d882d62; no voters:
I20260811 09:13:49.946012 6002 leader_election.cc:288] T 00000000000000000000000000000000 P 7ea76ecc11bd4a90920b73f82d882d62 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 09:13:49.946151 6027 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7ea76ecc11bd4a90920b73f82d882d62 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:13:49.946837 6027 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7ea76ecc11bd4a90920b73f82d882d62 [term 1 LEADER]: Becoming Leader. State: Replica: 7ea76ecc11bd4a90920b73f82d882d62, State: Running, Role: LEADER
I20260811 09:13:49.947391 6027 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7ea76ecc11bd4a90920b73f82d882d62 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ea76ecc11bd4a90920b73f82d882d62" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 33293 } }
I20260811 09:13:49.948181 6002 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7ea76ecc11bd4a90920b73f82d882d62 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:13:49.950791 6029 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ea76ecc11bd4a90920b73f82d882d62 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7ea76ecc11bd4a90920b73f82d882d62. Latest consensus state: current_term: 1 leader_uuid: "7ea76ecc11bd4a90920b73f82d882d62" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ea76ecc11bd4a90920b73f82d882d62" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 33293 } } }
I20260811 09:13:49.951282 6029 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ea76ecc11bd4a90920b73f82d882d62 [sys.catalog]: This master's current role is: LEADER
I20260811 09:13:49.950851 6028 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ea76ecc11bd4a90920b73f82d882d62 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7ea76ecc11bd4a90920b73f82d882d62" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ea76ecc11bd4a90920b73f82d882d62" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 33293 } } }
I20260811 09:13:49.952914 6028 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ea76ecc11bd4a90920b73f82d882d62 [sys.catalog]: This master's current role is: LEADER
I20260811 09:13:49.954025 6039 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:13:49.959438 6039 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:13:49.968583 6039 catalog_manager.cc:1347] Generated new cluster ID: 177a0b7ea8ab495cae7012e0dfc7f22b
I20260811 09:13:49.968713 6039 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:13:49.991303 6039 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:13:49.991832 6039 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:13:50.002261 6039 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7ea76ecc11bd4a90920b73f82d882d62: Generated new TSK 0
I20260811 09:13:50.003391 6039 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 09:13:50.021965 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 5886
W20260811 09:13:50.023590 5994 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:42765
W20260811 09:13:50.024313 29290 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 5886: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260811 09:13:50.024508 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 5886
2026-08-11T09:13:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (2211 ms)
[----------] 18 tests from AuthTokenIssuingTest (40128 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439250192320-29290-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:13:50 dist-test-slave-xt4k krb5kdc[6058](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:13:50 dist-test-slave-xt4k krb5kdc[6058](info): set up 2 sockets
Aug 11 09:13:50 dist-test-slave-xt4k krb5kdc[6058](info): commencing operation
krb5kdc: starting...
W20260811 09:13:51.331611 29290 mini_kdc.cc:121] Time spent starting KDC: real 1.263s user 0.004s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439250192320-29290-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439250192320-29290-0/krb5kdc/test-user.keytab.
Aug 11 09:13:51 dist-test-slave-xt4k krb5kdc[6058](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439631, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:13:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:13:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.154.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.154.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.154.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab.
Entry for principal kudu/127.28.154.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab.
I20260811 09:13:51.675865 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439250192320-29290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439250192320-29290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439250192320-29290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.154.190:34179
--webserver_interface=127.28.154.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab
--principal=kudu/127.28.154.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:39173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.154.190:34179,127.28.154.189:37459,127.28.154.188:43121
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:13:52.023976 6079 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 09:13:52.024416 6079 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:13:52.024523 6079 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:13:52.024613 6079 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:13:52.032583 6079 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:13:52.032699 6079 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:13:52.032761 6079 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:13:52.032806 6079 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:13:52.032871 6079 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:13:52.043350 6079 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:39173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439250192320-29290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439250192320-29290-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.154.190:34179,127.28.154.189:37459,127.28.154.188:43121
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab
--principal=kudu/127.28.154.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.154.190:34179
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.154.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439250192320-29290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:13:52.044749 6079 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:13:52.046124 6079 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:13:52.058868 6092 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:13:52.058804 6101 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:13:52.058825 6094 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:13:52.059782 6079 server_base.cc:1034] running on GCE node
I20260811 09:13:52.060848 6079 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:13:52.062422 6079 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:13:52.063711 6079 hybrid_clock.cc:648] HybridClock initialized: now 1786439632063618 us; error 57 us; skew 500 ppm
I20260811 09:13:52.064148 6079 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:13:52 dist-test-slave-xt4k krb5kdc[6058](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439632, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:13:52.068065 6079 init.cc:377] Logged in from keytab as kudu/127.28.154.190@KRBTEST.COM (short username kudu)
I20260811 09:13:52.069764 6079 webserver.cc:460] Webserver started at http://127.28.154.190:40413/ using document root <none> and password file <none>
I20260811 09:13:52.070544 6079 fs_manager.cc:359] Metadata directory not provided
I20260811 09:13:52.070657 6079 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:13:52.071003 6079 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:13:52.074870 6079 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439250192320-29290-0/minicluster-data/master-0/data/instance:
uuid: "9187717a67d74ddd985ed960c9156b1b"
format_stamp: "Formatted at 2026-08-11 09:13:52 on dist-test-slave-xt4k"
server_key: "423fc10adcfcec6667b224ec34387c02"
server_key_iv: "68fd7cf2df3d04e0ae0b6b05b5b8baba"
server_key_version: "encryptionkey@0"
I20260811 09:13:52.075783 6079 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439250192320-29290-0/minicluster-data/master-0/wal/instance:
uuid: "9187717a67d74ddd985ed960c9156b1b"
format_stamp: "Formatted at 2026-08-11 09:13:52 on dist-test-slave-xt4k"
server_key: "423fc10adcfcec6667b224ec34387c02"
server_key_iv: "68fd7cf2df3d04e0ae0b6b05b5b8baba"
server_key_version: "encryptionkey@0"
I20260811 09:13:52.081807 6079 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260811 09:13:52.085644 6111 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:13:52.087057 6079 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 09:13:52.087242 6079 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439250192320-29290-0/minicluster-data/master-0/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439250192320-29290-0/minicluster-data/master-0/wal
uuid: "9187717a67d74ddd985ed960c9156b1b"
format_stamp: "Formatted at 2026-08-11 09:13:52 on dist-test-slave-xt4k"
server_key: "423fc10adcfcec6667b224ec34387c02"
server_key_iv: "68fd7cf2df3d04e0ae0b6b05b5b8baba"
server_key_version: "encryptionkey@0"
I20260811 09:13:52.087469 6079 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439250192320-29290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439250192320-29290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439250192320-29290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:13:52.106063 6079 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:13:52.110788 6079 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:13:52.111080 6079 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:13:52.130671 6079 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.190:34179
I20260811 09:13:52.130756 6214 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.190:34179 every 8 connection(s)
I20260811 09:13:52.132314 6079 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
I20260811 09:13:52.136591 6216 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:13:52.139662 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 6079
I20260811 09:13:52.139890 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439250192320-29290-0/minicluster-data/master-0/wal/instance
I20260811 09:13:52.140591 29290 external_mini_cluster.cc:1428] Setting key 6815eb20f6d6c64c4d980ec61e125628
I20260811 09:13:52.143890 6216 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.154.190" port: 34179 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 09:13:52 dist-test-slave-xt4k krb5kdc[6058](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439632, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.190@KRBTEST.COM for kudu/127.28.154.190@KRBTEST.COM
I20260811 09:13:52.167313 6221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:52.148904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59178 (local address 127.28.154.190:34179)
0811 09:13:52.149604 (+ 700us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:52.149614 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:52.149672 (+ 58us) server_negotiation.cc:407] Connection header received
0811 09:13:52.150969 (+ 1297us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:52.150975 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:52.151227 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:52.151404 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:52.152809 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:52.154036 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:52.155402 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:52.155977 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:52.156153 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:52.160095 (+ 3942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:13:52.160105 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:13:52.160120 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:13:52.160152 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:13:52.162940 (+ 2788us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:13:52.163821 (+ 881us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:13:52.163829 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:13:52.163831 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:13:52.164008 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:13:52.164497 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:13:52.164501 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:13:52.164503 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:13:52.165088 (+ 585us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:13:52.165258 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:52.165954 (+ 696us) server_negotiation.cc:299] Negotiation successful
0811 09:13:52.166224 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":476,"thread_start_us":304,"threads_started":1}
I20260811 09:13:52.167447 6220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:52.147476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:34179 (local address 127.0.0.1:59178)
0811 09:13:52.148396 (+ 920us) negotiation.cc:107] Waiting for socket to connect
0811 09:13:52.148450 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 09:13:52.150591 (+ 2141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:13:52.151454 (+ 863us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:13:52.151460 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:13:52.152186 (+ 726us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:13:52.152586 (+ 400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:13:52.152608 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:13:52.154183 (+ 1575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:13:52.154186 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:13:52.155266 (+ 1080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:13:52.155280 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:13:52.156228 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:13:52.156235 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:13:52.156472 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:52.157143 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:13:52.157166 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:13:52.159813 (+ 2647us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:13:52.163092 (+ 3279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:13:52.163106 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:13:52.163120 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:13:52.163651 (+ 531us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:13:52.164131 (+ 480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:13:52.164135 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:13:52.164138 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:13:52.164369 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:13:52.165253 (+ 884us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:13:52.165263 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:13:52.165527 (+ 264us) client_negotiation.cc:769] Sending connection context
0811 09:13:52.165786 (+ 259us) client_negotiation.cc:240] Negotiation successful
0811 09:13:52.166422 (+ 636us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":591,"mutex_wait_us":133,"thread_start_us":360,"threads_started":1}
I20260811 09:13:52.170833 6216 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.154.189" port: 37459 } has no permanent_uuid. Determining permanent_uuid...
I20260811 09:13:52.172092 6220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:52.171621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.189:37459 (local address 127.0.0.1:42712)
0811 09:13:52.171819 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0811 09:13:52.171955 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.189:37459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
WARNING: no policy specified for kudu/127.28.154.189@KRBTEST.COM; defaulting to no policy
W20260811 09:13:52.172441 6119 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.28.154.189:37459: connect: Connection refused (error 111)
I20260811 09:13:52.174239 6220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:52.173794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.189:37459 (local address 127.0.0.1:42718)
0811 09:13:52.173962 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0811 09:13:52.174045 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.189:37459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260811 09:13:52.174505 6216 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.154.189:37459: Network error: Client connection negotiation failed: client connection to 127.28.154.189:37459: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.154.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.154.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.189.keytab.
Entry for principal kudu/127.28.154.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.189.keytab.
I20260811 09:13:52.211611 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439250192320-29290-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439250192320-29290-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439250192320-29290-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439250192320-29290-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.154.189:37459
--webserver_interface=127.28.154.189
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.189.keytab
--principal=kudu/127.28.154.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:39173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.154.190:34179,127.28.154.189:37459,127.28.154.188:43121
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 09:13:52.223932 6216 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.154.189" port: 37459 } attempt: 1
I20260811 09:13:52.225144 6220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:52.224674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.189:37459 (local address 127.0.0.1:42724)
0811 09:13:52.224875 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0811 09:13:52.225008 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.189:37459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260811 09:13:52.226233 6220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:52.225851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.189:37459 (local address 127.0.0.1:42736)
0811 09:13:52.226013 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0811 09:13:52.226092 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.189:37459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260811 09:13:52.226440 6216 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.154.189:37459: Network error: Client connection negotiation failed: client connection to 127.28.154.189:37459: connect: Connection refused (error 111)
I20260811 09:13:52.294723 6216 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.154.189" port: 37459 } attempt: 2
I20260811 09:13:52.295850 6220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:52.295373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.189:37459 (local address 127.0.0.1:42744)
0811 09:13:52.295612 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0811 09:13:52.295703 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.189:37459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260811 09:13:52.297104 6220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:52.296663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.189:37459 (local address 127.0.0.1:42760)
0811 09:13:52.296857 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0811 09:13:52.296940 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.189:37459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260811 09:13:52.297307 6216 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.154.189:37459: Network error: Client connection negotiation failed: client connection to 127.28.154.189:37459: connect: Connection refused (error 111)
I20260811 09:13:52.388561 6216 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.154.189" port: 37459 } attempt: 3
I20260811 09:13:52.389840 6220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:52.389366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.189:37459 (local address 127.0.0.1:42776)
0811 09:13:52.389587 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0811 09:13:52.389704 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.189:37459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260811 09:13:52.390929 6220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:52.390527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.189:37459 (local address 127.0.0.1:42778)
0811 09:13:52.390679 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0811 09:13:52.390779 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.189:37459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260811 09:13:52.391147 6216 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.154.189:37459: Network error: Client connection negotiation failed: client connection to 127.28.154.189:37459: connect: Connection refused (error 111)
I20260811 09:13:52.534420 6216 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.154.189" port: 37459 } attempt: 4
I20260811 09:13:52.535732 6220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:52.535141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.189:37459 (local address 127.0.0.1:42792)
0811 09:13:52.535481 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0811 09:13:52.535585 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.189:37459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183}
I20260811 09:13:52.536849 6220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:52.536475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.189:37459 (local address 127.0.0.1:42806)
0811 09:13:52.536626 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0811 09:13:52.536717 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.189:37459: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260811 09:13:52.537052 6216 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.154.189:37459: Network error: Client connection negotiation failed: client connection to 127.28.154.189:37459: connect: Connection refused (error 111)
W20260811 09:13:52.560153 6234 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 09:13:52.560540 6234 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:13:52.560619 6234 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:13:52.560702 6234 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:13:52.568151 6234 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:13:52.568251 6234 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:13:52.568310 6234 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:13:52.568397 6234 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:13:52.568473 6234 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:13:52.579070 6234 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:39173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439250192320-29290-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439250192320-29290-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.28.154.190:34179,127.28.154.189:37459,127.28.154.188:43121
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.189.keytab
--principal=kudu/127.28.154.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.154.189:37459
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439250192320-29290-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.28.154.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439250192320-29290-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:13:52.580430 6234 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:13:52.581705 6234 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:13:52.593323 6252 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:13:52.593498 6250 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:13:52.593691 6255 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:13:52.594162 6234 server_base.cc:1034] running on GCE node
I20260811 09:13:52.594882 6234 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:13:52.596288 6234 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:13:52.597523 6234 hybrid_clock.cc:648] HybridClock initialized: now 1786439632597446 us; error 55 us; skew 500 ppm
I20260811 09:13:52.597846 6234 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:13:52 dist-test-slave-xt4k krb5kdc[6058](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439632, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:13:52.600975 6234 init.cc:377] Logged in from keytab as kudu/127.28.154.189@KRBTEST.COM (short username kudu)
I20260811 09:13:52.602468 6234 webserver.cc:460] Webserver started at http://127.28.154.189:41313/ using document root <none> and password file <none>
I20260811 09:13:52.603168 6234 fs_manager.cc:359] Metadata directory not provided
I20260811 09:13:52.603268 6234 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:13:52.603571 6234 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:13:52.606531 6234 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439250192320-29290-0/minicluster-data/master-1/data/instance:
uuid: "f8902506e4824570b3175fd3e3de8f03"
format_stamp: "Formatted at 2026-08-11 09:13:52 on dist-test-slave-xt4k"
server_key: "12918ed6a4e7bc41bafb346f3310898f"
server_key_iv: "7f55bd25504c2197e273587dae96a084"
server_key_version: "encryptionkey@0"
I20260811 09:13:52.607321 6234 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439250192320-29290-0/minicluster-data/master-1/wal/instance:
uuid: "f8902506e4824570b3175fd3e3de8f03"
format_stamp: "Formatted at 2026-08-11 09:13:52 on dist-test-slave-xt4k"
server_key: "12918ed6a4e7bc41bafb346f3310898f"
server_key_iv: "7f55bd25504c2197e273587dae96a084"
server_key_version: "encryptionkey@0"
I20260811 09:13:52.612565 6234 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260811 09:13:52.616021 6266 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:13:52.617406 6234 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 09:13:52.617578 6234 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439250192320-29290-0/minicluster-data/master-1/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439250192320-29290-0/minicluster-data/master-1/wal
uuid: "f8902506e4824570b3175fd3e3de8f03"
format_stamp: "Formatted at 2026-08-11 09:13:52 on dist-test-slave-xt4k"
server_key: "12918ed6a4e7bc41bafb346f3310898f"
server_key_iv: "7f55bd25504c2197e273587dae96a084"
server_key_version: "encryptionkey@0"
I20260811 09:13:52.617777 6234 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439250192320-29290-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439250192320-29290-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439250192320-29290-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:13:52.643430 6234 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:13:52.648077 6234 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:13:52.648394 6234 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:13:52.667088 6234 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.189:37459
I20260811 09:13:52.667106 6364 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.189:37459 every 8 connection(s)
I20260811 09:13:52.668529 6234 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439250192320-29290-0/minicluster-data/master-1/data/info.pb
I20260811 09:13:52.672396 6365 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:13:52.675549 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 6234
I20260811 09:13:52.675758 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439250192320-29290-0/minicluster-data/master-1/wal/instance
I20260811 09:13:52.676296 29290 external_mini_cluster.cc:1428] Setting key 38bba4fc8ecd966b90d11e45193aa3a5
I20260811 09:13:52.680933 6365 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.154.190" port: 34179 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 09:13:52 dist-test-slave-xt4k krb5kdc[6058](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439632, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.189@KRBTEST.COM for kudu/127.28.154.190@KRBTEST.COM
I20260811 09:13:52.702164 6369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:52.684155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59186 (local address 127.28.154.190:34179)
0811 09:13:52.684696 (+ 541us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:52.684702 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:52.685751 (+ 1049us) server_negotiation.cc:407] Connection header received
0811 09:13:52.687675 (+ 1924us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:52.687680 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:52.687764 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:52.687886 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:52.689540 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:52.690518 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:52.692225 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:52.692709 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:52.692840 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:52.696283 (+ 3443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:13:52.696288 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:13:52.696291 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:13:52.696367 (+ 76us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:13:52.698991 (+ 2624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:13:52.699761 (+ 770us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:13:52.699766 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:13:52.699768 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:13:52.699892 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:13:52.700306 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:13:52.700310 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:13:52.700312 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:13:52.700649 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:13:52.700791 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:52.701534 (+ 743us) server_negotiation.cc:299] Negotiation successful
0811 09:13:52.701771 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":414,"thread_start_us":298,"threads_started":1}
I20260811 09:13:52.703258 6370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:52.684764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:34179 (local address 127.0.0.1:59186)
0811 09:13:52.685575 (+ 811us) negotiation.cc:107] Waiting for socket to connect
0811 09:13:52.685620 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 09:13:52.687388 (+ 1768us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:13:52.688077 (+ 689us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:13:52.688085 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:13:52.688721 (+ 636us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:13:52.689312 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:13:52.689330 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:13:52.690706 (+ 1376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:13:52.690710 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:13:52.692079 (+ 1369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:13:52.692097 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:13:52.692933 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:13:52.692940 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:13:52.693182 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:52.693826 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:13:52.693850 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:13:52.696043 (+ 2193us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:13:52.699144 (+ 3101us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:13:52.699154 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:13:52.699175 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:13:52.699638 (+ 463us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:13:52.699999 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:13:52.700005 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:13:52.700009 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:13:52.700192 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:13:52.700791 (+ 599us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:13:52.700800 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:13:52.701169 (+ 369us) client_negotiation.cc:769] Sending connection context
0811 09:13:52.701433 (+ 264us) client_negotiation.cc:240] Negotiation successful
0811 09:13:52.702008 (+ 575us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":516,"thread_start_us":314,"threads_started":1}
I20260811 09:13:52.705104 6365 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.154.189" port: 37459 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.28.154.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.154.188@KRBTEST.COM" created.
Aug 11 09:13:52 dist-test-slave-xt4k krb5kdc[6058](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439632, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.189@KRBTEST.COM for kudu/127.28.154.189@KRBTEST.COM
I20260811 09:13:52.720122 6370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:52.705918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.189:37459 (local address 127.0.0.1:42816)
0811 09:13:52.706115 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0811 09:13:52.706131 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 09:13:52.706220 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:13:52.707239 (+ 1019us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:13:52.707244 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:13:52.707666 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:13:52.707807 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:13:52.707816 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:13:52.709272 (+ 1456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:13:52.709276 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:13:52.710317 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:13:52.710332 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:13:52.711238 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:13:52.711244 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:13:52.711403 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:52.712001 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:13:52.712023 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:13:52.714467 (+ 2444us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:13:52.717666 (+ 3199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:13:52.717671 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:13:52.717674 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:13:52.718122 (+ 448us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:13:52.718451 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:13:52.718455 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:13:52.718458 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:13:52.718629 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:13:52.719204 (+ 575us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:13:52.719208 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:13:52.719355 (+ 147us) client_negotiation.cc:769] Sending connection context
0811 09:13:52.719492 (+ 137us) client_negotiation.cc:240] Negotiation successful
0811 09:13:52.719755 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260811 09:13:52.720158 6373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:52.706127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42816 (local address 127.28.154.189:37459)
0811 09:13:52.706810 (+ 683us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:52.706816 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:52.706842 (+ 26us) server_negotiation.cc:407] Connection header received
0811 09:13:52.706890 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:52.706896 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:52.707100 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:52.707239 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:52.707968 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:52.709140 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:52.710481 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:52.711060 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:52.711166 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:52.714709 (+ 3543us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:13:52.714723 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:13:52.714741 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:13:52.714772 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:13:52.717534 (+ 2762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:13:52.718238 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:13:52.718245 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:13:52.718247 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:13:52.718360 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:13:52.718735 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:13:52.718739 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:13:52.718741 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:13:52.719090 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:13:52.719257 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:52.719642 (+ 385us) server_negotiation.cc:299] Negotiation successful
0811 09:13:52.719851 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":543,"thread_start_us":408,"threads_started":1}
I20260811 09:13:52.722585 6365 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.154.188" port: 43121 } has no permanent_uuid. Determining permanent_uuid...
I20260811 09:13:52.723649 6370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:52.723197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.188:43121 (local address 127.0.0.1:42504)
0811 09:13:52.723357 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0811 09:13:52.723490 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.188:43121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260811 09:13:52.723874 6278 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.28.154.188:43121: connect: Connection refused (error 111)
I20260811 09:13:52.725472 6370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:52.725154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.188:43121 (local address 127.0.0.1:42520)
0811 09:13:52.725280 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0811 09:13:52.725360 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.188:43121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260811 09:13:52.725705 6365 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.154.188:43121: Network error: Client connection negotiation failed: client connection to 127.28.154.188:43121: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.154.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.188.keytab.
Entry for principal kudu/127.28.154.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.188.keytab.
I20260811 09:13:52.749784 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439250192320-29290-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439250192320-29290-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439250192320-29290-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439250192320-29290-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.154.188:43121
--webserver_interface=127.28.154.188
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.188.keytab
--principal=kudu/127.28.154.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:39173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.154.190:34179,127.28.154.189:37459,127.28.154.188:43121
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 09:13:52.775053 6365 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.154.188" port: 43121 } attempt: 1
I20260811 09:13:52.776530 6370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:52.775935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.188:43121 (local address 127.0.0.1:42522)
0811 09:13:52.776177 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0811 09:13:52.776311 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.188:43121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260811 09:13:52.777750 6370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:52.777326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.188:43121 (local address 127.0.0.1:42534)
0811 09:13:52.777500 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0811 09:13:52.777602 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.188:43121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260811 09:13:52.778002 6365 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.154.188:43121: Network error: Client connection negotiation failed: client connection to 127.28.154.188:43121: connect: Connection refused (error 111)
I20260811 09:13:52.836324 6216 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.154.189" port: 37459 } attempt: 5
Aug 11 09:13:52 dist-test-slave-xt4k krb5kdc[6058](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439632, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.190@KRBTEST.COM for kudu/127.28.154.189@KRBTEST.COM
I20260811 09:13:52.846232 6365 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.154.188" port: 43121 } attempt: 2
I20260811 09:13:52.847398 6370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:52.846888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.188:43121 (local address 127.0.0.1:42544)
0811 09:13:52.847101 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0811 09:13:52.847204 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.188:43121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260811 09:13:52.848603 6370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:52.848170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.188:43121 (local address 127.0.0.1:42556)
0811 09:13:52.848325 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0811 09:13:52.848434 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.188:43121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260811 09:13:52.848874 6365 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.154.188:43121: Network error: Client connection negotiation failed: client connection to 127.28.154.188:43121: connect: Connection refused (error 111)
I20260811 09:13:52.851351 6373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:52.837438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42828 (local address 127.28.154.189:37459)
0811 09:13:52.837637 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:52.837644 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:52.837668 (+ 24us) server_negotiation.cc:407] Connection header received
0811 09:13:52.837757 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:52.837762 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:52.837867 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:52.837973 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:52.838758 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:52.839866 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:52.841189 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:52.841911 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:52.842054 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:52.845491 (+ 3437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:13:52.845496 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:13:52.845500 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:13:52.845536 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:13:52.848771 (+ 3235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:13:52.849475 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:13:52.849481 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:13:52.849485 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:13:52.849611 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:13:52.849972 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:13:52.849977 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:13:52.849981 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:13:52.850277 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:13:52.850412 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:52.850816 (+ 404us) server_negotiation.cc:299] Negotiation successful
0811 09:13:52.851039 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 09:13:52.851368 6220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:52.837273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.189:37459 (local address 127.0.0.1:42828)
0811 09:13:52.837494 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0811 09:13:52.837517 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 09:13:52.837640 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:13:52.838007 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:13:52.838010 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:13:52.838405 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:13:52.838593 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:13:52.838603 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:13:52.840031 (+ 1428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:13:52.840036 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:13:52.841020 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:13:52.841036 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:13:52.842103 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:13:52.842108 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:13:52.842273 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:52.842923 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:13:52.842946 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:13:52.845244 (+ 2298us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:13:52.848931 (+ 3687us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:13:52.848937 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:13:52.848942 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:13:52.849342 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:13:52.849721 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:13:52.849726 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:13:52.849730 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:13:52.849867 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:13:52.850412 (+ 545us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:13:52.850417 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:13:52.850563 (+ 146us) client_negotiation.cc:769] Sending connection context
0811 09:13:52.850692 (+ 129us) client_negotiation.cc:240] Negotiation successful
0811 09:13:52.851025 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":72}
I20260811 09:13:52.852756 6216 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.154.188" port: 43121 } has no permanent_uuid. Determining permanent_uuid...
I20260811 09:13:52.853822 6220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:52.853435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.188:43121 (local address 127.0.0.1:42560)
0811 09:13:52.853604 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0811 09:13:52.853686 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.188:43121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260811 09:13:52.854846 6220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:52.854514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.188:43121 (local address 127.0.0.1:42570)
0811 09:13:52.854671 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0811 09:13:52.854746 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.188:43121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260811 09:13:52.855072 6216 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.154.188:43121: Network error: Client connection negotiation failed: client connection to 127.28.154.188:43121: connect: Connection refused (error 111)
I20260811 09:13:52.906306 6216 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.154.188" port: 43121 } attempt: 1
I20260811 09:13:52.907621 6220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:52.907092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.188:43121 (local address 127.0.0.1:42576)
0811 09:13:52.907330 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0811 09:13:52.907438 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.188:43121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260811 09:13:52.908758 6220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:52.908302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.188:43121 (local address 127.0.0.1:42580)
0811 09:13:52.908518 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0811 09:13:52.908612 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.188:43121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
W20260811 09:13:52.909030 6216 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.154.188:43121: Network error: Client connection negotiation failed: client connection to 127.28.154.188:43121: connect: Connection refused (error 111)
I20260811 09:13:52.940124 6365 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.154.188" port: 43121 } attempt: 3
I20260811 09:13:52.941447 6370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:52.940892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.188:43121 (local address 127.0.0.1:42594)
0811 09:13:52.941125 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0811 09:13:52.941240 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.188:43121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260811 09:13:52.942842 6370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:52.942256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.188:43121 (local address 127.0.0.1:42610)
0811 09:13:52.942547 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0811 09:13:52.942663 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.188:43121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155}
W20260811 09:13:52.943137 6365 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.154.188:43121: Network error: Client connection negotiation failed: client connection to 127.28.154.188:43121: connect: Connection refused (error 111)
I20260811 09:13:52.977303 6216 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.154.188" port: 43121 } attempt: 2
I20260811 09:13:52.978772 6220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:52.978105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.188:43121 (local address 127.0.0.1:42612)
0811 09:13:52.978431 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0811 09:13:52.978556 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.188:43121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260811 09:13:52.979872 6220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:52.979471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.188:43121 (local address 127.0.0.1:42616)
0811 09:13:52.979629 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0811 09:13:52.979726 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.188:43121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260811 09:13:52.980120 6216 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.154.188:43121: Network error: Client connection negotiation failed: client connection to 127.28.154.188:43121: connect: Connection refused (error 111)
I20260811 09:13:53.086365 6216 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.154.188" port: 43121 } attempt: 3
I20260811 09:13:53.086404 6365 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.154.188" port: 43121 } attempt: 4
I20260811 09:13:53.087592 6370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:53.087095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.188:43121 (local address 127.0.0.1:42632)
0811 09:13:53.087329 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0811 09:13:53.087427 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.188:43121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260811 09:13:53.087749 6220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:53.087120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.188:43121 (local address 127.0.0.1:42640)
0811 09:13:53.087403 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0811 09:13:53.087499 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.188:43121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163}
I20260811 09:13:53.088892 6370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:53.088314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.188:43121 (local address 127.0.0.1:42650)
0811 09:13:53.088523 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0811 09:13:53.088620 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.188:43121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101}
I20260811 09:13:53.088982 6220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:53.088413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.188:43121 (local address 127.0.0.1:42656)
0811 09:13:53.088620 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0811 09:13:53.088807 (+ 187us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.188:43121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"threads_started":1}
W20260811 09:13:53.089221 6365 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.154.188:43121: Network error: Client connection negotiation failed: client connection to 127.28.154.188:43121: connect: Connection refused (error 111)
W20260811 09:13:53.089887 6216 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.154.188:43121: Network error: Client connection negotiation failed: client connection to 127.28.154.188:43121: connect: Connection refused (error 111)
W20260811 09:13:53.107602 6380 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 09:13:53.108137 6380 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:13:53.108239 6380 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:13:53.108394 6380 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:13:53.116211 6380 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:13:53.116351 6380 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:13:53.116441 6380 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:13:53.116508 6380 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:13:53.116595 6380 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:13:53.127233 6380 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:39173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439250192320-29290-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439250192320-29290-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.28.154.190:34179,127.28.154.189:37459,127.28.154.188:43121
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.188.keytab
--principal=kudu/127.28.154.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.154.188:43121
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439250192320-29290-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.28.154.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439250192320-29290-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:13:53.128692 6380 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:13:53.130086 6380 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:13:53.143280 6393 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:13:53.143245 6390 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:13:53.143483 6389 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:13:53.144164 6380 server_base.cc:1034] running on GCE node
I20260811 09:13:53.145139 6380 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:13:53.146736 6380 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:13:53.147995 6380 hybrid_clock.cc:648] HybridClock initialized: now 1786439633147925 us; error 42 us; skew 500 ppm
I20260811 09:13:53.148412 6380 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:13:53 dist-test-slave-xt4k krb5kdc[6058](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439633, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:13:53.152215 6380 init.cc:377] Logged in from keytab as kudu/127.28.154.188@KRBTEST.COM (short username kudu)
I20260811 09:13:53.154110 6380 webserver.cc:460] Webserver started at http://127.28.154.188:38097/ using document root <none> and password file <none>
I20260811 09:13:53.154901 6380 fs_manager.cc:359] Metadata directory not provided
I20260811 09:13:53.155015 6380 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:13:53.155350 6380 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:13:53.158545 6380 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439250192320-29290-0/minicluster-data/master-2/data/instance:
uuid: "f2ca1c35f22b40d396dd80345abffb94"
format_stamp: "Formatted at 2026-08-11 09:13:53 on dist-test-slave-xt4k"
server_key: "59882efbc70fa41de11657c856123749"
server_key_iv: "a41dfeeff5d52b5d109d0d9af1677b9e"
server_key_version: "encryptionkey@0"
I20260811 09:13:53.159413 6380 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439250192320-29290-0/minicluster-data/master-2/wal/instance:
uuid: "f2ca1c35f22b40d396dd80345abffb94"
format_stamp: "Formatted at 2026-08-11 09:13:53 on dist-test-slave-xt4k"
server_key: "59882efbc70fa41de11657c856123749"
server_key_iv: "a41dfeeff5d52b5d109d0d9af1677b9e"
server_key_version: "encryptionkey@0"
I20260811 09:13:53.165627 6380 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.002s
I20260811 09:13:53.169440 6409 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:13:53.170909 6380 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260811 09:13:53.171099 6380 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439250192320-29290-0/minicluster-data/master-2/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439250192320-29290-0/minicluster-data/master-2/wal
uuid: "f2ca1c35f22b40d396dd80345abffb94"
format_stamp: "Formatted at 2026-08-11 09:13:53 on dist-test-slave-xt4k"
server_key: "59882efbc70fa41de11657c856123749"
server_key_iv: "a41dfeeff5d52b5d109d0d9af1677b9e"
server_key_version: "encryptionkey@0"
I20260811 09:13:53.171321 6380 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439250192320-29290-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439250192320-29290-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439250192320-29290-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:13:53.186247 6380 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:13:53.190899 6380 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:13:53.191218 6380 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:13:53.208525 6380 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.188:43121
I20260811 09:13:53.208549 6512 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.188:43121 every 8 connection(s)
I20260811 09:13:53.209940 6380 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439250192320-29290-0/minicluster-data/master-2/data/info.pb
I20260811 09:13:53.213996 6513 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:13:53.214697 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 6380
I20260811 09:13:53.214891 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786439250192320-29290-0/minicluster-data/master-2/wal/instance
I20260811 09:13:53.215508 29290 external_mini_cluster.cc:1428] Setting key 73a204d1ed258e37cb3c7de27c381d63
I20260811 09:13:53.224061 6513 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.154.190" port: 34179 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 09:13:53 dist-test-slave-xt4k krb5kdc[6058](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439631, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.154.188@KRBTEST.COM
Aug 11 09:13:53 dist-test-slave-xt4k krb5kdc[6058](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439633, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.188@KRBTEST.COM for kudu/127.28.154.190@KRBTEST.COM
I20260811 09:13:53.244243 6519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:53.217520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42670 (local address 127.28.154.188:43121)
0811 09:13:53.218298 (+ 778us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:53.218313 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:53.218357 (+ 44us) server_negotiation.cc:407] Connection header received
0811 09:13:53.220388 (+ 2031us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:53.220422 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:53.221047 (+ 625us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:53.221449 (+ 402us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:53.223664 (+ 2215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:53.225708 (+ 2044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:53.227642 (+ 1934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:53.228781 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:53.228966 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:53.235920 (+ 6954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:13:53.235929 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:13:53.235943 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:13:53.236044 (+ 101us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:13:53.239040 (+ 2996us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:13:53.239858 (+ 818us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:13:53.239865 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:13:53.239873 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:13:53.240037 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:13:53.240494 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:13:53.240499 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:13:53.240503 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:13:53.241315 (+ 812us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:13:53.241638 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:53.242234 (+ 596us) server_negotiation.cc:299] Negotiation successful
0811 09:13:53.242766 (+ 532us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":573,"spinlock_wait_cycles":18176,"thread_start_us":352,"threads_started":1}
I20260811 09:13:53.245445 6523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:53.227593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59188 (local address 127.28.154.190:34179)
0811 09:13:53.229760 (+ 2167us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:53.229767 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:53.229789 (+ 22us) server_negotiation.cc:407] Connection header received
0811 09:13:53.229854 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:53.229859 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:53.229935 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:53.230043 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:53.231465 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:53.232379 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:53.234177 (+ 1798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:53.234827 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:53.234960 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:53.238872 (+ 3912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:13:53.238878 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:13:53.238881 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:13:53.238917 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:13:53.241893 (+ 2976us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:13:53.242948 (+ 1055us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:13:53.242953 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:13:53.242957 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:13:53.243175 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:13:53.243613 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:13:53.243618 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:13:53.243621 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:13:53.243998 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:13:53.244183 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:53.244811 (+ 628us) server_negotiation.cc:299] Negotiation successful
0811 09:13:53.245069 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2034,"thread_start_us":1829,"threads_started":1}
I20260811 09:13:53.245505 6522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:53.227479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:34179 (local address 127.0.0.1:59188)
0811 09:13:53.228966 (+ 1487us) negotiation.cc:107] Waiting for socket to connect
0811 09:13:53.229021 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 09:13:53.229157 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:13:53.230289 (+ 1132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:13:53.230295 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:13:53.231005 (+ 710us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:13:53.231296 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:13:53.231307 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:13:53.232542 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:13:53.232547 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:13:53.233997 (+ 1450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:13:53.234014 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:13:53.235088 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:13:53.235097 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:13:53.235304 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:53.236010 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:13:53.236029 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:13:53.238662 (+ 2633us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:13:53.242054 (+ 3392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:13:53.242066 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:13:53.242074 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:13:53.242807 (+ 733us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:13:53.243325 (+ 518us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:13:53.243329 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:13:53.243331 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:13:53.243494 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:13:53.244232 (+ 738us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:13:53.244239 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:13:53.244458 (+ 219us) client_negotiation.cc:769] Sending connection context
0811 09:13:53.244675 (+ 217us) client_negotiation.cc:240] Negotiation successful
0811 09:13:53.245026 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1203,"thread_start_us":1026,"threads_started":1}
I20260811 09:13:53.247028 6513 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.154.189" port: 37459 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 09:13:53 dist-test-slave-xt4k krb5kdc[6058](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439633, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.188@KRBTEST.COM for kudu/127.28.154.189@KRBTEST.COM
Aug 11 09:13:53 dist-test-slave-xt4k krb5kdc[6058](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439631, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.154.190@KRBTEST.COM
I20260811 09:13:53.262559 6522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:53.247996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.189:37459 (local address 127.0.0.1:42836)
0811 09:13:53.248163 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0811 09:13:53.248180 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 09:13:53.248269 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:13:53.248620 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:13:53.248625 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:13:53.249164 (+ 539us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:13:53.249386 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:13:53.249398 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:13:53.250746 (+ 1348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:13:53.250750 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:13:53.252099 (+ 1349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:13:53.252114 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:13:53.253130 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:13:53.253135 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:13:53.253303 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:53.253906 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:13:53.253930 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:13:53.256329 (+ 2399us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:13:53.259825 (+ 3496us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:13:53.259831 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:13:53.259835 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:13:53.260379 (+ 544us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:13:53.260767 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:13:53.260772 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:13:53.260775 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:13:53.260946 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:13:53.261611 (+ 665us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:13:53.261614 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:13:53.261778 (+ 164us) client_negotiation.cc:769] Sending connection context
0811 09:13:53.261933 (+ 155us) client_negotiation.cc:240] Negotiation successful
0811 09:13:53.262198 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260811 09:13:53.262576 6373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:53.248135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42836 (local address 127.28.154.189:37459)
0811 09:13:53.248293 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:53.248297 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:53.248331 (+ 34us) server_negotiation.cc:407] Connection header received
0811 09:13:53.248418 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:53.248424 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:53.248490 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:53.248560 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:53.249573 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:53.250587 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:53.252264 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:53.252952 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:53.253166 (+ 214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:53.256557 (+ 3391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:13:53.256563 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:13:53.256566 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:13:53.256602 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:13:53.259672 (+ 3070us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:13:53.260503 (+ 831us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:13:53.260508 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:13:53.260510 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:13:53.260658 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:13:53.261043 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:13:53.261045 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:13:53.261047 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:13:53.261478 (+ 431us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:13:53.261628 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:53.262032 (+ 404us) server_negotiation.cc:299] Negotiation successful
0811 09:13:53.262264 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 09:13:53.263463 6513 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.154.188" port: 43121 } has no permanent_uuid. Determining permanent_uuid...
I20260811 09:13:53.264070 6523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:53.247652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59190 (local address 127.28.154.190:34179)
0811 09:13:53.247867 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:53.247874 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:53.247895 (+ 21us) server_negotiation.cc:407] Connection header received
0811 09:13:53.247947 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:53.247951 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:53.248042 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:53.248145 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:53.249940 (+ 1795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:53.250877 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:53.252583 (+ 1706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:53.253381 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:53.253487 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:53.257965 (+ 4478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:13:53.257971 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:13:53.257974 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:13:53.258006 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:13:53.261005 (+ 2999us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:13:53.261806 (+ 801us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:13:53.261811 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:13:53.261815 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:13:53.261989 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:13:53.262411 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:13:53.262416 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:13:53.262419 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:13:53.262973 (+ 554us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:13:53.263235 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:53.263553 (+ 318us) server_negotiation.cc:299] Negotiation successful
0811 09:13:53.263768 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260811 09:13:53.267112 6216 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.154.188" port: 43121 } attempt: 4
Aug 11 09:13:53 dist-test-slave-xt4k krb5kdc[6058](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439633, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.188@KRBTEST.COM for kudu/127.28.154.188@KRBTEST.COM
Aug 11 09:13:53 dist-test-slave-xt4k krb5kdc[6058](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439631, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.154.189@KRBTEST.COM
Aug 11 09:13:53 dist-test-slave-xt4k krb5kdc[6058](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439632, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.190@KRBTEST.COM for kudu/127.28.154.188@KRBTEST.COM
I20260811 09:13:53.281515 6522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:53.264122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.188:43121 (local address 127.0.0.1:42672)
0811 09:13:53.264478 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0811 09:13:53.264492 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 09:13:53.264568 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:13:53.264834 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:13:53.264838 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:13:53.265285 (+ 447us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:13:53.265430 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:13:53.265439 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:13:53.266933 (+ 1494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:13:53.266940 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:13:53.268171 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:13:53.268186 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:13:53.269272 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:13:53.269276 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:13:53.269426 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:53.269950 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:13:53.269967 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:13:53.272607 (+ 2640us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:13:53.278758 (+ 6151us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:13:53.278763 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:13:53.278766 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:13:53.279267 (+ 501us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:13:53.279701 (+ 434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:13:53.279706 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:13:53.279710 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:13:53.279842 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:13:53.280439 (+ 597us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:13:53.280444 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:13:53.280601 (+ 157us) client_negotiation.cc:769] Sending connection context
0811 09:13:53.280721 (+ 120us) client_negotiation.cc:240] Negotiation successful
0811 09:13:53.281080 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"mutex_wait_us":3}
I20260811 09:13:53.281600 6519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:53.264330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42672 (local address 127.28.154.188:43121)
0811 09:13:53.264511 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:53.264516 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:53.264534 (+ 18us) server_negotiation.cc:407] Connection header received
0811 09:13:53.264673 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:53.264678 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:53.264731 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:53.264804 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:53.265602 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:53.266768 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:53.268308 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:53.268973 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:53.269062 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:53.272785 (+ 3723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:13:53.272791 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:13:53.272794 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:13:53.272822 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:13:53.278542 (+ 5720us) spinlock_profiling.cc:243] Waited 1.44 ms on lock 0x60c0000347b8. stack: 00007fe49fb2666e 00007fe49fb26430 00007fe49ebad5ed 00000000008ea064 00000000008ea02d 00000000008e99fa 00007fe49fb1c0fd 00007fe49fb1c977 00007fe4a663369d 00007fe4a664f963 00007fe4a664901e 00007fe4a66428a5 00007fe4a65b0ea5 00007fe4a65afeff 00007fe4a65e382f 00007fe4a65e358d
0811 09:13:53.278588 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:13:53.279411 (+ 823us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:13:53.279415 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:13:53.279423 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:13:53.279574 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:13:53.279943 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:13:53.279948 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:13:53.279952 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:13:53.280320 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:13:53.280481 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:53.280815 (+ 334us) server_negotiation.cc:299] Negotiation successful
0811 09:13:53.281251 (+ 436us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":194,"server-negotiator.queue_time_us":38,"spinlock_wait_cycles":3317632}
I20260811 09:13:53.283641 6373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:53.265836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42842 (local address 127.28.154.189:37459)
0811 09:13:53.265985 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:53.265989 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:53.266003 (+ 14us) server_negotiation.cc:407] Connection header received
0811 09:13:53.266153 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:53.266156 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:53.266213 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:53.266285 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:53.267880 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:53.268841 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:53.270401 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:53.270797 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:53.270912 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:53.276652 (+ 5740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:13:53.276664 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:13:53.276670 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:13:53.276715 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:13:53.280113 (+ 3398us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:13:53.280838 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:13:53.280844 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:13:53.280847 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:13:53.281020 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:13:53.281506 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:13:53.281511 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:13:53.281514 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:13:53.281848 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:13:53.282135 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:53.283108 (+ 973us) server_negotiation.cc:299] Negotiation successful
0811 09:13:53.283343 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 09:13:53.285854 6381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:53.267713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.188:43121 (local address 127.0.0.1:42682)
0811 09:13:53.267985 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0811 09:13:53.268004 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 09:13:53.268142 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:13:53.269054 (+ 912us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:13:53.269059 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:13:53.269649 (+ 590us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:13:53.269941 (+ 292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:13:53.269965 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:13:53.271787 (+ 1822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:13:53.271794 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:13:53.273306 (+ 1512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:13:53.273325 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:13:53.274421 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:13:53.274428 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:13:53.274633 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:53.275161 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:13:53.275182 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:13:53.277848 (+ 2666us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:13:53.282552 (+ 4704us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:13:53.282557 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:13:53.282560 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:13:53.283097 (+ 537us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:13:53.283492 (+ 395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:13:53.283497 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:13:53.283501 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:13:53.283672 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:13:53.284824 (+ 1152us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:13:53.284829 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:13:53.284997 (+ 168us) client_negotiation.cc:769] Sending connection context
0811 09:13:53.285171 (+ 174us) client_negotiation.cc:240] Negotiation successful
0811 09:13:53.285465 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126}
I20260811 09:13:53.285934 6525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:53.267883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42682 (local address 127.28.154.188:43121)
0811 09:13:53.268748 (+ 865us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:53.268754 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:53.268780 (+ 26us) server_negotiation.cc:407] Connection header received
0811 09:13:53.268825 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:53.268830 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:53.268908 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:53.269017 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:53.270119 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:53.271468 (+ 1349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:53.273501 (+ 2033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:53.274201 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:53.274912 (+ 711us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:53.278165 (+ 3253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:13:53.278172 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:13:53.278177 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:13:53.278214 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:13:53.282411 (+ 4197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:13:53.283206 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:13:53.283212 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:13:53.283215 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:13:53.283370 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:13:53.284216 (+ 846us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:13:53.284221 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:13:53.284224 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:13:53.284673 (+ 449us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:13:53.284819 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:53.285278 (+ 459us) server_negotiation.cc:299] Negotiation successful
0811 09:13:53.285485 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":739,"thread_start_us":548,"threads_started":1}
I20260811 09:13:53.288301 6513 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2ca1c35f22b40d396dd80345abffb94: Bootstrap starting.
I20260811 09:13:53.288554 29290 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 09:13:53.291599 6513 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f2ca1c35f22b40d396dd80345abffb94: Neither blocks nor log segments found. Creating new log.
I20260811 09:13:53.292927 6513 log.cc:824] T 00000000000000000000000000000000 P f2ca1c35f22b40d396dd80345abffb94: Log is configured to *not* fsync() on all Append() calls
I20260811 09:13:53.291206 6169 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "9187717a67d74ddd985ed960c9156b1b"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59190
I20260811 09:13:53.295784 6513 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2ca1c35f22b40d396dd80345abffb94: No bootstrap required, opened a new log
I20260811 09:13:53.297176 6216 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9187717a67d74ddd985ed960c9156b1b: Bootstrap starting.
I20260811 09:13:53.301208 6216 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9187717a67d74ddd985ed960c9156b1b: Neither blocks nor log segments found. Creating new log.
I20260811 09:13:53.302534 6216 log.cc:824] T 00000000000000000000000000000000 P 9187717a67d74ddd985ed960c9156b1b: Log is configured to *not* fsync() on all Append() calls
I20260811 09:13:53.303913 6513 raft_consensus.cc:348] T 00000000000000000000000000000000 P f2ca1c35f22b40d396dd80345abffb94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9187717a67d74ddd985ed960c9156b1b" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 34179 } } peers { permanent_uuid: "f8902506e4824570b3175fd3e3de8f03" member_type: VOTER last_known_addr { host: "127.28.154.189" port: 37459 } } peers { permanent_uuid: "f2ca1c35f22b40d396dd80345abffb94" member_type: VOTER last_known_addr { host: "127.28.154.188" port: 43121 } }
I20260811 09:13:53.304329 6513 raft_consensus.cc:729] T 00000000000000000000000000000000 P f2ca1c35f22b40d396dd80345abffb94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2ca1c35f22b40d396dd80345abffb94, State: Initialized, Role: FOLLOWER
I20260811 09:13:53.304905 6513 consensus_queue.cc:260] T 00000000000000000000000000000000 P f2ca1c35f22b40d396dd80345abffb94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9187717a67d74ddd985ed960c9156b1b" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 34179 } } peers { permanent_uuid: "f8902506e4824570b3175fd3e3de8f03" member_type: VOTER last_known_addr { host: "127.28.154.189" port: 37459 } } peers { permanent_uuid: "f2ca1c35f22b40d396dd80345abffb94" member_type: VOTER last_known_addr { host: "127.28.154.188" port: 43121 } }
I20260811 09:13:53.306121 6216 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9187717a67d74ddd985ed960c9156b1b: No bootstrap required, opened a new log
I20260811 09:13:53.305838 6530 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2ca1c35f22b40d396dd80345abffb94 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9187717a67d74ddd985ed960c9156b1b" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 34179 } } peers { permanent_uuid: "f8902506e4824570b3175fd3e3de8f03" member_type: VOTER last_known_addr { host: "127.28.154.189" port: 37459 } } peers { permanent_uuid: "f2ca1c35f22b40d396dd80345abffb94" member_type: VOTER last_known_addr { host: "127.28.154.188" port: 43121 } } }
I20260811 09:13:53.306386 6530 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2ca1c35f22b40d396dd80345abffb94 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 09:13:53.307165 6513 sys_catalog.cc:564] T 00000000000000000000000000000000 P f2ca1c35f22b40d396dd80345abffb94 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:13:53.313798 6216 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9187717a67d74ddd985ed960c9156b1b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9187717a67d74ddd985ed960c9156b1b" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 34179 } } peers { permanent_uuid: "f8902506e4824570b3175fd3e3de8f03" member_type: VOTER last_known_addr { host: "127.28.154.189" port: 37459 } } peers { permanent_uuid: "f2ca1c35f22b40d396dd80345abffb94" member_type: VOTER last_known_addr { host: "127.28.154.188" port: 43121 } }
I20260811 09:13:53.314260 6216 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9187717a67d74ddd985ed960c9156b1b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9187717a67d74ddd985ed960c9156b1b, State: Initialized, Role: FOLLOWER
I20260811 09:13:53.314791 6216 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9187717a67d74ddd985ed960c9156b1b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9187717a67d74ddd985ed960c9156b1b" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 34179 } } peers { permanent_uuid: "f8902506e4824570b3175fd3e3de8f03" member_type: VOTER last_known_addr { host: "127.28.154.189" port: 37459 } } peers { permanent_uuid: "f2ca1c35f22b40d396dd80345abffb94" member_type: VOTER last_known_addr { host: "127.28.154.188" port: 43121 } }
I20260811 09:13:53.315832 6554 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9187717a67d74ddd985ed960c9156b1b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9187717a67d74ddd985ed960c9156b1b" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 34179 } } peers { permanent_uuid: "f8902506e4824570b3175fd3e3de8f03" member_type: VOTER last_known_addr { host: "127.28.154.189" port: 37459 } } peers { permanent_uuid: "f2ca1c35f22b40d396dd80345abffb94" member_type: VOTER last_known_addr { host: "127.28.154.188" port: 43121 } } }
I20260811 09:13:53.316322 6554 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9187717a67d74ddd985ed960c9156b1b [sys.catalog]: This master's current role is: FOLLOWER
W20260811 09:13:53.316602 6549 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f2ca1c35f22b40d396dd80345abffb94: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 09:13:53.316699 6549 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 09:13:53.317143 6216 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9187717a67d74ddd985ed960c9156b1b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:13:53.320719 6169 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9187717a67d74ddd985ed960c9156b1b [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260811 09:13:53.321159 6169 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9187717a67d74ddd985ed960c9156b1b [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:13:53.325052 6169 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9187717a67d74ddd985ed960c9156b1b [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9187717a67d74ddd985ed960c9156b1b" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 34179 } } peers { permanent_uuid: "f8902506e4824570b3175fd3e3de8f03" member_type: VOTER last_known_addr { host: "127.28.154.189" port: 37459 } } peers { permanent_uuid: "f2ca1c35f22b40d396dd80345abffb94" member_type: VOTER last_known_addr { host: "127.28.154.188" port: 43121 } }
I20260811 09:13:53.314761 6380 master_runner.cc:186] Error getting master registration for 127.28.154.190:34179: OK, instance_id { permanent_uuid: "9187717a67d74ddd985ed960c9156b1b" instance_seqno: 1786439632124969 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 09:13:53.328394 6380 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 09:13:53.328943 6169 leader_election.cc:288] T 00000000000000000000000000000000 P 9187717a67d74ddd985ed960c9156b1b [CANDIDATE]: Term 1 election: Requested vote from peers f8902506e4824570b3175fd3e3de8f03 (127.28.154.189:37459), f2ca1c35f22b40d396dd80345abffb94 (127.28.154.188:43121)
I20260811 09:13:53.329443 6467 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9187717a67d74ddd985ed960c9156b1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "f2ca1c35f22b40d396dd80345abffb94"
I20260811 09:13:53.330209 6467 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f2ca1c35f22b40d396dd80345abffb94 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:13:53.334198 6467 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f2ca1c35f22b40d396dd80345abffb94 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9187717a67d74ddd985ed960c9156b1b in term 1.
I20260811 09:13:53.334981 6119 leader_election.cc:302] T 00000000000000000000000000000000 P 9187717a67d74ddd985ed960c9156b1b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9187717a67d74ddd985ed960c9156b1b, f2ca1c35f22b40d396dd80345abffb94; no voters:
I20260811 09:13:53.335600 6554 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9187717a67d74ddd985ed960c9156b1b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:13:53.346639 6554 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9187717a67d74ddd985ed960c9156b1b [term 1 LEADER]: Becoming Leader. State: Replica: 9187717a67d74ddd985ed960c9156b1b, State: Running, Role: LEADER
I20260811 09:13:53.347414 6554 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9187717a67d74ddd985ed960c9156b1b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9187717a67d74ddd985ed960c9156b1b" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 34179 } } peers { permanent_uuid: "f8902506e4824570b3175fd3e3de8f03" member_type: VOTER last_known_addr { host: "127.28.154.189" port: 37459 } } peers { permanent_uuid: "f2ca1c35f22b40d396dd80345abffb94" member_type: VOTER last_known_addr { host: "127.28.154.188" port: 43121 } }
I20260811 09:13:53.356158 6567 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9187717a67d74ddd985ed960c9156b1b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9187717a67d74ddd985ed960c9156b1b. Latest consensus state: current_term: 1 leader_uuid: "9187717a67d74ddd985ed960c9156b1b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9187717a67d74ddd985ed960c9156b1b" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 34179 } } peers { permanent_uuid: "f8902506e4824570b3175fd3e3de8f03" member_type: VOTER last_known_addr { host: "127.28.154.189" port: 37459 } } peers { permanent_uuid: "f2ca1c35f22b40d396dd80345abffb94" member_type: VOTER last_known_addr { host: "127.28.154.188" port: 43121 } } }
I20260811 09:13:53.356724 6567 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9187717a67d74ddd985ed960c9156b1b [sys.catalog]: This master's current role is: LEADER
W20260811 09:13:53.364528 6126 tablet.cc:2369] T 00000000000000000000000000000000 P 9187717a67d74ddd985ed960c9156b1b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 09:13:53.365536 6581 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:13:53.366128 6583 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 09:13:53.380194 6581 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:13:53.370997 6079 master_runner.cc:186] Error getting master registration for 127.28.154.189:37459: OK, instance_id { permanent_uuid: "f8902506e4824570b3175fd3e3de8f03" instance_seqno: 1786439632661953 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 09:13:53.385774 6079 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 09:13:53.388414 6365 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.154.188" port: 43121 } attempt: 5
Aug 11 09:13:53 dist-test-slave-xt4k krb5kdc[6058](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439632, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.189@KRBTEST.COM for kudu/127.28.154.188@KRBTEST.COM
I20260811 09:13:53.403086 6467 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f2ca1c35f22b40d396dd80345abffb94 [term 1 FOLLOWER]: Refusing update from remote peer 9187717a67d74ddd985ed960c9156b1b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:13:53.404490 6554 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9187717a67d74ddd985ed960c9156b1b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2ca1c35f22b40d396dd80345abffb94" member_type: VOTER last_known_addr { host: "127.28.154.188" port: 43121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:13:53.414338 6370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:53.389148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.188:43121 (local address 127.0.0.1:42698)
0811 09:13:53.389341 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0811 09:13:53.389362 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 09:13:53.389469 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:13:53.389991 (+ 522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:13:53.389996 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:13:53.390543 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:13:53.390758 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:13:53.390771 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:13:53.392230 (+ 1459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:13:53.392236 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:13:53.393784 (+ 1548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:13:53.393801 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:13:53.395574 (+ 1773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:13:53.395581 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:13:53.395793 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:53.397717 (+ 1924us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:13:53.397742 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:13:53.402278 (+ 4536us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:13:53.411201 (+ 8923us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:13:53.411207 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:13:53.411211 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:13:53.411751 (+ 540us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:13:53.412292 (+ 541us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:13:53.412297 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:13:53.412301 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:13:53.412538 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:13:53.413315 (+ 777us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:13:53.413321 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:13:53.413500 (+ 179us) client_negotiation.cc:769] Sending connection context
0811 09:13:53.413645 (+ 145us) client_negotiation.cc:240] Negotiation successful
0811 09:13:53.413975 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260811 09:13:53.418813 6519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:53.389359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42698 (local address 127.28.154.188:43121)
0811 09:13:53.389539 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:53.389545 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:53.389565 (+ 20us) server_negotiation.cc:407] Connection header received
0811 09:13:53.389626 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:53.389631 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:53.389706 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:53.389805 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:53.390935 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:53.392039 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:53.393978 (+ 1939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:53.394772 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:53.394896 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:53.404498 (+ 9602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:13:53.404506 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:13:53.404511 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:13:53.404555 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:13:53.410731 (+ 6176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:13:53.411926 (+ 1195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:13:53.411931 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:13:53.411935 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:13:53.412132 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:13:53.412683 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:13:53.412689 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:13:53.412692 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:13:53.413097 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:13:53.413247 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:53.415181 (+ 1934us) server_negotiation.cc:299] Negotiation successful
0811 09:13:53.418410 (+ 3229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":49}
I20260811 09:13:53.425011 6373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:53.370695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42856 (local address 127.28.154.189:37459)
0811 09:13:53.370878 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:53.370884 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:53.380970 (+ 10086us) server_negotiation.cc:407] Connection header received
0811 09:13:53.388687 (+ 7717us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:53.388694 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:53.388789 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:53.388915 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:53.400005 (+ 11090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:53.401244 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:53.408220 (+ 6976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:53.408974 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:53.409083 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:53.417849 (+ 8766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:13:53.417856 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:13:53.417860 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:13:53.417898 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:13:53.421658 (+ 3760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:13:53.422486 (+ 828us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:13:53.422491 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:13:53.422495 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:13:53.422663 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:13:53.423113 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:13:53.423118 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:13:53.423122 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:13:53.423487 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:13:53.423714 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:53.424395 (+ 681us) server_negotiation.cc:299] Negotiation successful
0811 09:13:53.424640 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 09:13:53.425402 6365 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8902506e4824570b3175fd3e3de8f03: Bootstrap starting.
I20260811 09:13:53.427588 6523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:53.370854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59200 (local address 127.28.154.190:34179)
0811 09:13:53.371030 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:53.371036 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:53.376303 (+ 5267us) server_negotiation.cc:407] Connection header received
0811 09:13:53.381501 (+ 5198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:53.381508 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:53.381598 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:53.381713 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:53.391235 (+ 9522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:53.392377 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:53.394575 (+ 2198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:53.395256 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:53.395355 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:53.420550 (+ 25195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:13:53.420559 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:13:53.420563 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:13:53.420613 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:13:53.424225 (+ 3612us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:13:53.425127 (+ 902us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:13:53.425133 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:13:53.425137 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:13:53.425294 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:13:53.425740 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:13:53.425746 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:13:53.425749 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:13:53.426152 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:13:53.426406 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:53.426976 (+ 570us) server_negotiation.cc:299] Negotiation successful
0811 09:13:53.427233 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 09:13:53.430009 6365 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f8902506e4824570b3175fd3e3de8f03: Neither blocks nor log segments found. Creating new log.
I20260811 09:13:53.430524 6525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:53.375453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42690 (local address 127.28.154.188:43121)
0811 09:13:53.375639 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:53.375645 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:53.389861 (+ 14216us) server_negotiation.cc:407] Connection header received
0811 09:13:53.390027 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:53.390032 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:53.390121 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:53.390215 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:53.393495 (+ 3280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:53.395380 (+ 1885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:53.409325 (+ 13945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:53.410101 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:53.410224 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:53.414278 (+ 4054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:13:53.414284 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:13:53.414289 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:13:53.414325 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:13:53.417644 (+ 3319us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:13:53.427457 (+ 9813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:13:53.427463 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:13:53.427467 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:13:53.427638 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:13:53.428100 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:13:53.428105 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:13:53.428108 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:13:53.428526 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:13:53.428744 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:53.429856 (+ 1112us) server_negotiation.cc:299] Negotiation successful
0811 09:13:53.430120 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":805,"server-negotiator.queue_time_us":72}
I20260811 09:13:53.431723 6365 log.cc:824] T 00000000000000000000000000000000 P f8902506e4824570b3175fd3e3de8f03: Log is configured to *not* fsync() on all Append() calls
I20260811 09:13:53.441543 6365 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f8902506e4824570b3175fd3e3de8f03: No bootstrap required, opened a new log
W20260811 09:13:53.443461 6426 tablet.cc:2369] T 00000000000000000000000000000000 P f2ca1c35f22b40d396dd80345abffb94: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 09:13:53.447294 6530 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2ca1c35f22b40d396dd80345abffb94 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9187717a67d74ddd985ed960c9156b1b. Latest consensus state: current_term: 1 leader_uuid: "9187717a67d74ddd985ed960c9156b1b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9187717a67d74ddd985ed960c9156b1b" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 34179 } } peers { permanent_uuid: "f8902506e4824570b3175fd3e3de8f03" member_type: VOTER last_known_addr { host: "127.28.154.189" port: 37459 } } peers { permanent_uuid: "f2ca1c35f22b40d396dd80345abffb94" member_type: VOTER last_known_addr { host: "127.28.154.188" port: 43121 } } }
I20260811 09:13:53.447618 6530 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2ca1c35f22b40d396dd80345abffb94 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 09:13:53.453635 6365 raft_consensus.cc:348] T 00000000000000000000000000000000 P f8902506e4824570b3175fd3e3de8f03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9187717a67d74ddd985ed960c9156b1b" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 34179 } } peers { permanent_uuid: "f8902506e4824570b3175fd3e3de8f03" member_type: VOTER last_known_addr { host: "127.28.154.189" port: 37459 } } peers { permanent_uuid: "f2ca1c35f22b40d396dd80345abffb94" member_type: VOTER last_known_addr { host: "127.28.154.188" port: 43121 } }
I20260811 09:13:53.454190 6365 raft_consensus.cc:729] T 00000000000000000000000000000000 P f8902506e4824570b3175fd3e3de8f03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8902506e4824570b3175fd3e3de8f03, State: Initialized, Role: FOLLOWER
I20260811 09:13:53.454779 6365 consensus_queue.cc:260] T 00000000000000000000000000000000 P f8902506e4824570b3175fd3e3de8f03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9187717a67d74ddd985ed960c9156b1b" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 34179 } } peers { permanent_uuid: "f8902506e4824570b3175fd3e3de8f03" member_type: VOTER last_known_addr { host: "127.28.154.189" port: 37459 } } peers { permanent_uuid: "f2ca1c35f22b40d396dd80345abffb94" member_type: VOTER last_known_addr { host: "127.28.154.188" port: 43121 } }
I20260811 09:13:53.475085 6554 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9187717a67d74ddd985ed960c9156b1b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9187717a67d74ddd985ed960c9156b1b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9187717a67d74ddd985ed960c9156b1b" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 34179 } } peers { permanent_uuid: "f8902506e4824570b3175fd3e3de8f03" member_type: VOTER last_known_addr { host: "127.28.154.189" port: 37459 } } peers { permanent_uuid: "f2ca1c35f22b40d396dd80345abffb94" member_type: VOTER last_known_addr { host: "127.28.154.188" port: 43121 } } }
I20260811 09:13:53.475452 6554 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9187717a67d74ddd985ed960c9156b1b [sys.catalog]: This master's current role is: LEADER
I20260811 09:13:53.484647 6581 catalog_manager.cc:1347] Generated new cluster ID: 8a8230465e704824bb73df7d74e47be7
I20260811 09:13:53.484926 6581 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:13:53.497332 6530 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2ca1c35f22b40d396dd80345abffb94 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9187717a67d74ddd985ed960c9156b1b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9187717a67d74ddd985ed960c9156b1b" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 34179 } } peers { permanent_uuid: "f8902506e4824570b3175fd3e3de8f03" member_type: VOTER last_known_addr { host: "127.28.154.189" port: 37459 } } peers { permanent_uuid: "f2ca1c35f22b40d396dd80345abffb94" member_type: VOTER last_known_addr { host: "127.28.154.188" port: 43121 } } }
I20260811 09:13:53.497628 6530 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2ca1c35f22b40d396dd80345abffb94 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 09:13:53.497974 6599 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8902506e4824570b3175fd3e3de8f03 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9187717a67d74ddd985ed960c9156b1b" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 34179 } } peers { permanent_uuid: "f8902506e4824570b3175fd3e3de8f03" member_type: VOTER last_known_addr { host: "127.28.154.189" port: 37459 } } peers { permanent_uuid: "f2ca1c35f22b40d396dd80345abffb94" member_type: VOTER last_known_addr { host: "127.28.154.188" port: 43121 } } }
I20260811 09:13:53.498584 6599 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8902506e4824570b3175fd3e3de8f03 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 09:13:53.499871 6365 sys_catalog.cc:564] T 00000000000000000000000000000000 P f8902506e4824570b3175fd3e3de8f03 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:13:53.499960 6318 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9187717a67d74ddd985ed960c9156b1b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "f8902506e4824570b3175fd3e3de8f03"
I20260811 09:13:53.500555 6318 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f8902506e4824570b3175fd3e3de8f03 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:13:53.505226 6318 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f8902506e4824570b3175fd3e3de8f03 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9187717a67d74ddd985ed960c9156b1b in term 1.
I20260811 09:13:53.506047 6317 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f8902506e4824570b3175fd3e3de8f03 [term 1 FOLLOWER]: Refusing update from remote peer 9187717a67d74ddd985ed960c9156b1b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:13:53.508306 6567 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9187717a67d74ddd985ed960c9156b1b [LEADER]: Connected to new peer: Peer: permanent_uuid: "f8902506e4824570b3175fd3e3de8f03" member_type: VOTER last_known_addr { host: "127.28.154.189" port: 37459 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:13:53.508520 6581 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:13:53.510663 6581 catalog_manager.cc:1504] Loading token signing keys...
I20260811 09:13:53.536617 6581 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9187717a67d74ddd985ed960c9156b1b: Generated new TSK 0
I20260811 09:13:53.549943 6581 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 09:13:53.579903 6599 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8902506e4824570b3175fd3e3de8f03 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9187717a67d74ddd985ed960c9156b1b. Latest consensus state: current_term: 1 leader_uuid: "9187717a67d74ddd985ed960c9156b1b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9187717a67d74ddd985ed960c9156b1b" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 34179 } } peers { permanent_uuid: "f8902506e4824570b3175fd3e3de8f03" member_type: VOTER last_known_addr { host: "127.28.154.189" port: 37459 } } peers { permanent_uuid: "f2ca1c35f22b40d396dd80345abffb94" member_type: VOTER last_known_addr { host: "127.28.154.188" port: 43121 } } }
I20260811 09:13:53.580269 6599 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8902506e4824570b3175fd3e3de8f03 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 09:13:53.580902 6620 sys_catalog.cc:455] T 00000000000000000000000000000000 P f8902506e4824570b3175fd3e3de8f03 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9187717a67d74ddd985ed960c9156b1b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9187717a67d74ddd985ed960c9156b1b" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 34179 } } peers { permanent_uuid: "f8902506e4824570b3175fd3e3de8f03" member_type: VOTER last_known_addr { host: "127.28.154.189" port: 37459 } } peers { permanent_uuid: "f2ca1c35f22b40d396dd80345abffb94" member_type: VOTER last_known_addr { host: "127.28.154.188" port: 43121 } } }
I20260811 09:13:53.581377 6620 sys_catalog.cc:458] T 00000000000000000000000000000000 P f8902506e4824570b3175fd3e3de8f03 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 09:13:53.582813 6554 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9187717a67d74ddd985ed960c9156b1b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9187717a67d74ddd985ed960c9156b1b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9187717a67d74ddd985ed960c9156b1b" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 34179 } } peers { permanent_uuid: "f8902506e4824570b3175fd3e3de8f03" member_type: VOTER last_known_addr { host: "127.28.154.189" port: 37459 } } peers { permanent_uuid: "f2ca1c35f22b40d396dd80345abffb94" member_type: VOTER last_known_addr { host: "127.28.154.188" port: 43121 } } }
I20260811 09:13:53.583125 6554 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9187717a67d74ddd985ed960c9156b1b [sys.catalog]: This master's current role is: LEADER
I20260811 09:13:53.604547 6633 catalog_manager.cc:1259] Loaded cluster ID: 8a8230465e704824bb73df7d74e47be7
I20260811 09:13:53.604671 6633 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f8902506e4824570b3175fd3e3de8f03: loading cluster ID for follower catalog manager: success
I20260811 09:13:53.619323 6633 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f8902506e4824570b3175fd3e3de8f03: acquiring CA information for follower catalog manager: success
I20260811 09:13:53.622241 6633 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f8902506e4824570b3175fd3e3de8f03: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260811 09:13:53.705312 6373 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:13:53.695457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42860 (local address 127.28.154.189:37459)
0811 09:13:53.695694 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:53.695701 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:53.701762 (+ 6061us) server_negotiation.cc:407] Connection header received
0811 09:13:53.703516 (+ 1754us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:53.703522 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:53.703620 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:53.703709 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:53.705050 (+ 1341us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42860: BlockingRecv error: recv got EOF from 127.0.0.1:42860 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
W20260811 09:13:53.708935 6523 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:13:53.686336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59216 (local address 127.28.154.190:34179)
0811 09:13:53.686600 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:53.686607 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:53.701241 (+ 14634us) server_negotiation.cc:407] Connection header received
0811 09:13:53.707157 (+ 5916us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:53.707164 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:53.707271 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:53.707356 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:53.708684 (+ 1328us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59216: BlockingRecv error: recv got EOF from 127.0.0.1:59216 (error 108)
Metrics: {"server-negotiator.queue_time_us":99}
W20260811 09:13:53.712073 6525 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:13:53.702394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42712 (local address 127.28.154.188:43121)
0811 09:13:53.702584 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:53.702591 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:53.710266 (+ 7675us) server_negotiation.cc:407] Connection header received
0811 09:13:53.710626 (+ 360us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:53.710633 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:53.710725 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:53.710799 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:53.711786 (+ 987us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42712: BlockingRecv error: recv got EOF from 127.0.0.1:42712 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 09:13:53.763783 6373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:53.748130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42868 (local address 127.28.154.189:37459)
0811 09:13:53.748409 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:53.748416 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:53.755413 (+ 6997us) server_negotiation.cc:407] Connection header received
0811 09:13:53.756071 (+ 658us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:53.756080 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:53.756331 (+ 251us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:53.756510 (+ 179us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:13:53.757301 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:53.758459 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:53.760492 (+ 2033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:53.761176 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:53.761285 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:53.762116 (+ 831us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:13:53.762916 (+ 800us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:13:53.763067 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:53.763328 (+ 261us) server_negotiation.cc:299] Negotiation successful
0811 09:13:53.763446 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
W20260811 09:13:53.810774 6525 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:13:53.800707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42726 (local address 127.28.154.188:43121)
0811 09:13:53.800929 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:53.800935 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:53.808427 (+ 7492us) server_negotiation.cc:407] Connection header received
0811 09:13:53.808820 (+ 393us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:53.808828 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:53.809000 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:53.809106 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:53.810479 (+ 1373us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42726: BlockingRecv error: recv got EOF from 127.0.0.1:42726 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
I20260811 09:13:53.860096 6373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:53.845533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42882 (local address 127.28.154.189:37459)
0811 09:13:53.845788 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:53.845794 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:53.852765 (+ 6971us) server_negotiation.cc:407] Connection header received
0811 09:13:53.853340 (+ 575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:53.853346 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:53.853476 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:53.853548 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:13:53.854302 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:53.855561 (+ 1259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:53.857645 (+ 2084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:53.858354 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:53.858467 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:53.859151 (+ 684us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:13:53.859323 (+ 172us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:13:53.859387 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:53.859640 (+ 253us) server_negotiation.cc:299] Negotiation successful
0811 09:13:53.859739 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
W20260811 09:13:53.903509 6525 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:13:53.894318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42738 (local address 127.28.154.188:43121)
0811 09:13:53.894545 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:53.894553 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:53.901452 (+ 6899us) server_negotiation.cc:407] Connection header received
0811 09:13:53.901988 (+ 536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:53.901996 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:53.902102 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:53.902174 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:53.903228 (+ 1054us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42738: BlockingRecv error: recv got EOF from 127.0.0.1:42738 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 09:13:53.937674 6373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:53.929498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42898 (local address 127.28.154.189:37459)
0811 09:13:53.929697 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:53.929704 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:53.932921 (+ 3217us) server_negotiation.cc:407] Connection header received
0811 09:13:53.933135 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:53.933140 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:53.933212 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:53.933294 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:13:53.933765 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:53.934544 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:53.935883 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:53.936365 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:53.936497 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:53.936885 (+ 388us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:13:53.937023 (+ 138us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:13:53.937101 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:53.937250 (+ 149us) server_negotiation.cc:299] Negotiation successful
0811 09:13:53.937336 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
W20260811 09:13:53.946254 6525 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:13:53.943712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42746 (local address 127.28.154.188:43121)
0811 09:13:53.943894 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:53.943900 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:53.944862 (+ 962us) server_negotiation.cc:407] Connection header received
0811 09:13:53.945056 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:53.945062 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:53.945150 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:53.945219 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:53.946082 (+ 863us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42746: BlockingRecv error: recv got EOF from 127.0.0.1:42746 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 09:13:53.959460 6373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:53.953736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42908 (local address 127.28.154.189:37459)
0811 09:13:53.953950 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:53.953956 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:53.954566 (+ 610us) server_negotiation.cc:407] Connection header received
0811 09:13:53.954774 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:53.954779 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:53.954877 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:53.954956 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:13:53.955423 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:53.956214 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:53.957612 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:53.958123 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:53.958242 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:53.958648 (+ 406us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:13:53.958762 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:13:53.958839 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:53.959006 (+ 167us) server_negotiation.cc:299] Negotiation successful
0811 09:13:53.959119 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
W20260811 09:13:53.967703 6525 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:13:53.965552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42756 (local address 127.28.154.188:43121)
0811 09:13:53.965819 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:53.965826 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:53.966330 (+ 504us) server_negotiation.cc:407] Connection header received
0811 09:13:53.966540 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:53.966544 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:53.966617 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:53.966698 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:53.967451 (+ 753us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42756: BlockingRecv error: recv got EOF from 127.0.0.1:42756 (error 108)
Metrics: {"server-negotiator.queue_time_us":132}
I20260811 09:13:53.984113 6373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:53.978706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42910 (local address 127.28.154.189:37459)
0811 09:13:53.978919 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:53.978926 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:53.979450 (+ 524us) server_negotiation.cc:407] Connection header received
0811 09:13:53.979651 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:53.979657 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:53.979730 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:53.979806 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:13:53.980209 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:53.981060 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:53.982385 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:53.982858 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:53.982971 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:53.983396 (+ 425us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:13:53.983509 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:13:53.983584 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:53.983732 (+ 148us) server_negotiation.cc:299] Negotiation successful
0811 09:13:53.983815 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
W20260811 09:13:53.996558 6525 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:13:53.994297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42766 (local address 127.28.154.188:43121)
0811 09:13:53.994505 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:53.994513 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:53.995040 (+ 527us) server_negotiation.cc:407] Connection header received
0811 09:13:53.995225 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:53.995231 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:53.995322 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:53.995409 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:53.996291 (+ 882us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42766: BlockingRecv error: recv got EOF from 127.0.0.1:42766 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 09:13:54.024165 6373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:54.018284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42914 (local address 127.28.154.189:37459)
0811 09:13:54.018498 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:54.018505 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:54.019004 (+ 499us) server_negotiation.cc:407] Connection header received
0811 09:13:54.019213 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:54.019218 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:54.019297 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:54.019382 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:13:54.019801 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:54.020729 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:54.022268 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:54.022843 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:54.022960 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:54.023429 (+ 469us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:13:54.023558 (+ 129us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:13:54.023633 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:54.023782 (+ 149us) server_negotiation.cc:299] Negotiation successful
0811 09:13:54.023882 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
W20260811 09:13:54.032637 6525 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:13:54.030576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42774 (local address 127.28.154.188:43121)
0811 09:13:54.030783 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:54.030790 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:54.031279 (+ 489us) server_negotiation.cc:407] Connection header received
0811 09:13:54.031439 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:54.031445 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:54.031526 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:54.031613 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:54.032402 (+ 789us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42774: BlockingRecv error: recv got EOF from 127.0.0.1:42774 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 09:13:54.073287 6373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:54.067721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42918 (local address 127.28.154.189:37459)
0811 09:13:54.067990 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:54.068002 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:54.068588 (+ 586us) server_negotiation.cc:407] Connection header received
0811 09:13:54.068801 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:54.068805 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:54.068876 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:54.068961 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:13:54.069396 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:54.070145 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:54.071439 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:54.071917 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:54.072038 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:54.072492 (+ 454us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:13:54.072611 (+ 119us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:13:54.072692 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:54.072845 (+ 153us) server_negotiation.cc:299] Negotiation successful
0811 09:13:54.072931 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
W20260811 09:13:54.088486 6525 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:13:54.085817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42776 (local address 127.28.154.188:43121)
0811 09:13:54.086037 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:54.086044 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:54.086585 (+ 541us) server_negotiation.cc:407] Connection header received
0811 09:13:54.086829 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:54.086836 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:54.086955 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:54.087038 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:54.088215 (+ 1177us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42776: BlockingRecv error: recv got EOF from 127.0.0.1:42776 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 09:13:54.162956 6373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:54.157230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42920 (local address 127.28.154.189:37459)
0811 09:13:54.157443 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:54.157449 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:54.157892 (+ 443us) server_negotiation.cc:407] Connection header received
0811 09:13:54.158047 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:54.158052 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:54.158127 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:54.158226 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:13:54.158731 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:54.159495 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:54.161067 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:54.161542 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:54.161652 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:54.162069 (+ 417us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:13:54.162203 (+ 134us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:13:54.162293 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:54.162563 (+ 270us) server_negotiation.cc:299] Negotiation successful
0811 09:13:54.162641 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
W20260811 09:13:54.172159 6525 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:13:54.169963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42786 (local address 127.28.154.188:43121)
0811 09:13:54.170323 (+ 360us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:54.170329 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:54.170611 (+ 282us) server_negotiation.cc:407] Connection header received
0811 09:13:54.170935 (+ 324us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:54.170941 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:54.171018 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:54.171108 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:54.171916 (+ 808us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42786: BlockingRecv error: recv got EOF from 127.0.0.1:42786 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
I20260811 09:13:54.310077 6373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:54.305038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42922 (local address 127.28.154.189:37459)
0811 09:13:54.305283 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:54.305288 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:54.305420 (+ 132us) server_negotiation.cc:407] Connection header received
0811 09:13:54.305585 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:54.305590 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:54.305662 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:54.305741 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:13:54.306066 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:54.306774 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:54.308276 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:54.308768 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:54.308888 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:54.309359 (+ 471us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:13:54.309489 (+ 130us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:13:54.309570 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:54.309724 (+ 154us) server_negotiation.cc:299] Negotiation successful
0811 09:13:54.309812 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
W20260811 09:13:54.317826 6525 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:13:54.315903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42796 (local address 127.28.154.188:43121)
0811 09:13:54.316141 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:54.316147 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:54.316379 (+ 232us) server_negotiation.cc:407] Connection header received
0811 09:13:54.316594 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:54.316598 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:54.316670 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:54.316756 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:54.317563 (+ 807us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42796: BlockingRecv error: recv got EOF from 127.0.0.1:42796 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 09:13:54.319113 6549 catalog_manager.cc:1259] Loaded cluster ID: 8a8230465e704824bb73df7d74e47be7
I20260811 09:13:54.319192 6549 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f2ca1c35f22b40d396dd80345abffb94: loading cluster ID for follower catalog manager: success
I20260811 09:13:54.321858 6549 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f2ca1c35f22b40d396dd80345abffb94: acquiring CA information for follower catalog manager: success
I20260811 09:13:54.323722 6549 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f2ca1c35f22b40d396dd80345abffb94: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 09:13:54.582828 6373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:54.577855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42924 (local address 127.28.154.189:37459)
0811 09:13:54.578034 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:54.578039 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:54.578301 (+ 262us) server_negotiation.cc:407] Connection header received
0811 09:13:54.578474 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:54.578478 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:54.578548 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:54.578635 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:13:54.579123 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:54.579930 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:54.581335 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:54.581780 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:54.581875 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:54.582246 (+ 371us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:13:54.582369 (+ 123us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:13:54.582432 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:54.582549 (+ 117us) server_negotiation.cc:299] Negotiation successful
0811 09:13:54.582617 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 09:13:54.594096 6525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:54.588738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42798 (local address 127.28.154.188:43121)
0811 09:13:54.588941 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:54.588947 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:54.589101 (+ 154us) server_negotiation.cc:407] Connection header received
0811 09:13:54.589293 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:54.589297 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:54.589441 (+ 144us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:54.589539 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:13:54.589922 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:54.590664 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:54.591980 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:54.592430 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:54.592535 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:54.593023 (+ 488us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:13:54.593513 (+ 490us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:13:54.593648 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:54.593752 (+ 104us) server_negotiation.cc:299] Negotiation successful
0811 09:13:54.593842 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 09:13:54.608176 6373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:54.600596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42932 (local address 127.28.154.189:37459)
0811 09:13:54.600838 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:54.600844 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:54.601948 (+ 1104us) server_negotiation.cc:407] Connection header received
0811 09:13:54.602245 (+ 297us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:54.602252 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:54.602351 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:54.602436 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:13:54.603166 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:54.604184 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:54.605904 (+ 1720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:54.606478 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:54.606611 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:54.607310 (+ 699us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:13:54.607446 (+ 136us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:13:54.607530 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:54.607698 (+ 168us) server_negotiation.cc:299] Negotiation successful
0811 09:13:54.607791 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260811 09:13:54.610046 6525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:54.602069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42806 (local address 127.28.154.188:43121)
0811 09:13:54.602245 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:54.602252 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:54.603605 (+ 1353us) server_negotiation.cc:407] Connection header received
0811 09:13:54.603853 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:54.603859 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:54.603957 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:54.604058 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:13:54.604865 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:54.605988 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:54.607963 (+ 1975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:54.608584 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:54.608681 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:54.609285 (+ 604us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:13:54.609415 (+ 130us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:13:54.609492 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:54.609667 (+ 175us) server_negotiation.cc:299] Negotiation successful
0811 09:13:54.609746 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 09:13:54.610823 6858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:54.600517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59222 (local address 127.28.154.190:34179)
0811 09:13:54.603766 (+ 3249us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:54.603772 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:54.603791 (+ 19us) server_negotiation.cc:407] Connection header received
0811 09:13:54.603896 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:54.603900 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:54.604069 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:54.604267 (+ 198us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:13:54.605040 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:54.606097 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:54.608082 (+ 1985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:54.608782 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:54.609465 (+ 683us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:54.609704 (+ 239us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:13:54.610207 (+ 503us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:13:54.610310 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:54.610431 (+ 121us) server_negotiation.cc:299] Negotiation successful
0811 09:13:54.610525 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3128,"thread_start_us":3016,"threads_started":1}
I20260811 09:13:54.615226 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 6079
I20260811 09:13:54.616839 6209 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:13:54.726997 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 6079
W20260811 09:13:54.756242 6272 connection.cc:535] server connection from 127.0.0.1:42828 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 09:13:54.756309 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 6234
I20260811 09:13:54.757499 6359 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:13:54.870260 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 6234
I20260811 09:13:54.897948 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 6380
I20260811 09:13:54.899088 6502 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:13:55.008118 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 6380
2026-08-11T09:13:55Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (4980 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439250192320-29290-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 09:13:55 dist-test-slave-xt4k krb5kdc[6876](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 09:13:55 dist-test-slave-xt4k krb5kdc[6876](info): set up 2 sockets
Aug 11 09:13:55 dist-test-slave-xt4k krb5kdc[6876](info): commencing operation
krb5kdc: starting...
W20260811 09:13:56.316916 29290 mini_kdc.cc:121] Time spent starting KDC: real 1.268s user 0.001s sys 0.051s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439250192320-29290-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439250192320-29290-0/krb5kdc/test-user.keytab.
Aug 11 09:13:56 dist-test-slave-xt4k krb5kdc[6876](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439636, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T09:13:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T09:13:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.154.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.154.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.154.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab.
Entry for principal kudu/127.28.154.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab.
I20260811 09:13:56.665939 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439250192320-29290-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439250192320-29290-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439250192320-29290-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.154.190:44423
--webserver_interface=127.28.154.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab
--principal=kudu/127.28.154.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:39893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.154.190:44423,127.28.154.189:45629,127.28.154.188:37749
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 09:13:57.015486 6902 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 09:13:57.015892 6902 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:13:57.015998 6902 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:13:57.016126 6902 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:13:57.024022 6902 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:13:57.024165 6902 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:13:57.024277 6902 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:13:57.024379 6902 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:13:57.024482 6902 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:13:57.034785 6902 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:39893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439250192320-29290-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439250192320-29290-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.154.190:44423,127.28.154.189:45629,127.28.154.188:37749
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.190.keytab
--principal=kudu/127.28.154.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.154.190:44423
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.154.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439250192320-29290-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:13:57.036276 6902 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:13:57.037653 6902 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:13:57.049959 6911 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:13:57.050055 6916 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:13:57.050091 6914 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:13:57.051379 6902 server_base.cc:1034] running on GCE node
I20260811 09:13:57.052198 6902 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:13:57.053721 6902 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:13:57.054991 6902 hybrid_clock.cc:648] HybridClock initialized: now 1786439637054914 us; error 42 us; skew 500 ppm
I20260811 09:13:57.055333 6902 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:13:57 dist-test-slave-xt4k krb5kdc[6876](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439637, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:13:57.058516 6902 init.cc:377] Logged in from keytab as kudu/127.28.154.190@KRBTEST.COM (short username kudu)
I20260811 09:13:57.060016 6902 webserver.cc:460] Webserver started at http://127.28.154.190:43683/ using document root <none> and password file <none>
I20260811 09:13:57.060761 6902 fs_manager.cc:359] Metadata directory not provided
I20260811 09:13:57.060854 6902 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:13:57.061151 6902 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:13:57.064227 6902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439250192320-29290-0/minicluster-data/master-0/data/instance:
uuid: "d3ad7aeb39cf4daeadc0a36faa88710e"
format_stamp: "Formatted at 2026-08-11 09:13:57 on dist-test-slave-xt4k"
server_key: "9b5bd44b46c00a3186f9cd7bc0517771"
server_key_iv: "3d38278ea514dc3e106c1295e3eb37f0"
server_key_version: "encryptionkey@0"
I20260811 09:13:57.065061 6902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439250192320-29290-0/minicluster-data/master-0/wal/instance:
uuid: "d3ad7aeb39cf4daeadc0a36faa88710e"
format_stamp: "Formatted at 2026-08-11 09:13:57 on dist-test-slave-xt4k"
server_key: "9b5bd44b46c00a3186f9cd7bc0517771"
server_key_iv: "3d38278ea514dc3e106c1295e3eb37f0"
server_key_version: "encryptionkey@0"
I20260811 09:13:57.070395 6902 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 09:13:57.074071 6926 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:13:57.075587 6902 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 09:13:57.075841 6902 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439250192320-29290-0/minicluster-data/master-0/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439250192320-29290-0/minicluster-data/master-0/wal
uuid: "d3ad7aeb39cf4daeadc0a36faa88710e"
format_stamp: "Formatted at 2026-08-11 09:13:57 on dist-test-slave-xt4k"
server_key: "9b5bd44b46c00a3186f9cd7bc0517771"
server_key_iv: "3d38278ea514dc3e106c1295e3eb37f0"
server_key_version: "encryptionkey@0"
I20260811 09:13:57.076085 6902 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439250192320-29290-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439250192320-29290-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439250192320-29290-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:13:57.098834 6902 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:13:57.107370 6902 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:13:57.107705 6902 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:13:57.132069 6902 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.190:44423
I20260811 09:13:57.132097 7030 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.190:44423 every 8 connection(s)
I20260811 09:13:57.133872 6902 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439250192320-29290-0/minicluster-data/master-0/data/info.pb
I20260811 09:13:57.139052 7031 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:13:57.141628 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 6902
I20260811 09:13:57.141831 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439250192320-29290-0/minicluster-data/master-0/wal/instance
I20260811 09:13:57.142277 29290 external_mini_cluster.cc:1428] Setting key b171fe616cea201bacd3e751ea7b5d5b
I20260811 09:13:57.148027 7031 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.154.190" port: 44423 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 09:13:57 dist-test-slave-xt4k krb5kdc[6876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439637, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.190@KRBTEST.COM for kudu/127.28.154.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.28.154.189@KRBTEST.COM; defaulting to no policy
I20260811 09:13:57.174691 7033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:57.151540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:44423 (local address 127.0.0.1:57634)
0811 09:13:57.152384 (+ 844us) negotiation.cc:107] Waiting for socket to connect
0811 09:13:57.152435 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 09:13:57.154483 (+ 2048us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:13:57.157592 (+ 3109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:13:57.157599 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:13:57.158346 (+ 747us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:13:57.158802 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:13:57.158827 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:13:57.160474 (+ 1647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:13:57.160479 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:13:57.162024 (+ 1545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:13:57.162042 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:13:57.163108 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:13:57.163115 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:13:57.163300 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:57.164081 (+ 781us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:13:57.164110 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:13:57.166945 (+ 2835us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:13:57.170106 (+ 3161us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:13:57.170118 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:13:57.170131 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:13:57.170662 (+ 531us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:13:57.171162 (+ 500us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:13:57.171168 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:13:57.171172 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:13:57.171336 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:13:57.172175 (+ 839us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:13:57.172183 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:13:57.172440 (+ 257us) client_negotiation.cc:769] Sending connection context
0811 09:13:57.172675 (+ 235us) client_negotiation.cc:240] Negotiation successful
0811 09:13:57.173351 (+ 676us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":560,"mutex_wait_us":160,"thread_start_us":336,"threads_started":1}
I20260811 09:13:57.174710 7036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:57.156269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57634 (local address 127.28.154.190:44423)
0811 09:13:57.156914 (+ 645us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:57.156921 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:57.156952 (+ 31us) server_negotiation.cc:407] Connection header received
0811 09:13:57.157084 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:57.157089 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:57.157317 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:57.157480 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:57.159048 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:57.160279 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:57.162210 (+ 1931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:57.162839 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:57.162992 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:57.167194 (+ 4202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:13:57.167203 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:13:57.167217 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:13:57.167248 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:13:57.169967 (+ 2719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:13:57.170814 (+ 847us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:13:57.170833 (+ 19us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:13:57.170839 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:13:57.171014 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:13:57.171457 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:13:57.171461 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:13:57.171463 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:13:57.172011 (+ 548us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:13:57.172207 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:57.172820 (+ 613us) server_negotiation.cc:299] Negotiation successful
0811 09:13:57.173211 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":61,"server-negotiator.queue_time_us":479,"thread_start_us":343,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.154.189@KRBTEST.COM" created.
I20260811 09:13:57.178020 7031 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.154.189" port: 45629 } has no permanent_uuid. Determining permanent_uuid...
I20260811 09:13:57.179098 7033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:57.178620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.189:45629 (local address 127.0.0.1:41784)
0811 09:13:57.178808 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0811 09:13:57.178950 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.189:45629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260811 09:13:57.179370 6936 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.28.154.189:45629: connect: Connection refused (error 111)
I20260811 09:13:57.183873 7033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:57.183434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.189:45629 (local address 127.0.0.1:41790)
0811 09:13:57.183600 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0811 09:13:57.183690 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.189:45629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260811 09:13:57.184115 7031 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.154.189:45629: Network error: Client connection negotiation failed: client connection to 127.28.154.189:45629: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.154.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.189.keytab.
Entry for principal kudu/127.28.154.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.189.keytab.
I20260811 09:13:57.215441 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439250192320-29290-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439250192320-29290-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439250192320-29290-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439250192320-29290-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.154.189:45629
--webserver_interface=127.28.154.189
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.189.keytab
--principal=kudu/127.28.154.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:39893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.154.190:44423,127.28.154.189:45629,127.28.154.188:37749
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 09:13:57.233510 7031 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.154.189" port: 45629 } attempt: 1
I20260811 09:13:57.234692 7033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:57.234206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.189:45629 (local address 127.0.0.1:41802)
0811 09:13:57.234407 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0811 09:13:57.234535 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.189:45629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260811 09:13:57.235759 7033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:57.235377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.189:45629 (local address 127.0.0.1:41812)
0811 09:13:57.235556 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0811 09:13:57.235626 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.189:45629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260811 09:13:57.235949 7031 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.154.189:45629: Network error: Client connection negotiation failed: client connection to 127.28.154.189:45629: connect: Connection refused (error 111)
I20260811 09:13:57.304215 7031 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.154.189" port: 45629 } attempt: 2
I20260811 09:13:57.305450 7033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:57.304949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.189:45629 (local address 127.0.0.1:41826)
0811 09:13:57.305214 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0811 09:13:57.305307 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.189:45629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260811 09:13:57.306535 7033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:57.306167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.189:45629 (local address 127.0.0.1:41828)
0811 09:13:57.306329 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0811 09:13:57.306399 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.189:45629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260811 09:13:57.306730 7031 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.154.189:45629: Network error: Client connection negotiation failed: client connection to 127.28.154.189:45629: connect: Connection refused (error 111)
I20260811 09:13:57.398015 7031 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.154.189" port: 45629 } attempt: 3
I20260811 09:13:57.399288 7033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:57.398784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.189:45629 (local address 127.0.0.1:41842)
0811 09:13:57.399040 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0811 09:13:57.399139 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.189:45629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114}
I20260811 09:13:57.400475 7033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:57.400043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.189:45629 (local address 127.0.0.1:41846)
0811 09:13:57.400200 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0811 09:13:57.400274 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.189:45629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260811 09:13:57.400651 7031 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.154.189:45629: Network error: Client connection negotiation failed: client connection to 127.28.154.189:45629: connect: Connection refused (error 111)
I20260811 09:13:57.543926 7031 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.154.189" port: 45629 } attempt: 4
I20260811 09:13:57.545246 7033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:57.544690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.189:45629 (local address 127.0.0.1:41862)
0811 09:13:57.545010 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0811 09:13:57.545103 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.189:45629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170}
I20260811 09:13:57.546365 7033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:57.545936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.189:45629 (local address 127.0.0.1:41878)
0811 09:13:57.546126 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0811 09:13:57.546231 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.189:45629: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260811 09:13:57.546567 7031 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.154.189:45629: Network error: Client connection negotiation failed: client connection to 127.28.154.189:45629: connect: Connection refused (error 111)
W20260811 09:13:57.561697 7044 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 09:13:57.562045 7044 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:13:57.562137 7044 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:13:57.562248 7044 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:13:57.569634 7044 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:13:57.569738 7044 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:13:57.569831 7044 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:13:57.569904 7044 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:13:57.570001 7044 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:13:57.580178 7044 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:39893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439250192320-29290-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439250192320-29290-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.28.154.190:44423,127.28.154.189:45629,127.28.154.188:37749
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.189.keytab
--principal=kudu/127.28.154.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.154.189:45629
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439250192320-29290-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.28.154.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439250192320-29290-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:13:57.581494 7044 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:13:57.583027 7044 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:13:57.594537 7057 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:13:57.594589 7055 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:13:57.594612 7059 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:13:57.595335 7044 server_base.cc:1034] running on GCE node
I20260811 09:13:57.596230 7044 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:13:57.597674 7044 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:13:57.598892 7044 hybrid_clock.cc:648] HybridClock initialized: now 1786439637598822 us; error 44 us; skew 500 ppm
I20260811 09:13:57.599224 7044 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:13:57 dist-test-slave-xt4k krb5kdc[6876](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439637, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:13:57.602223 7044 init.cc:377] Logged in from keytab as kudu/127.28.154.189@KRBTEST.COM (short username kudu)
I20260811 09:13:57.603753 7044 webserver.cc:460] Webserver started at http://127.28.154.189:41131/ using document root <none> and password file <none>
I20260811 09:13:57.604452 7044 fs_manager.cc:359] Metadata directory not provided
I20260811 09:13:57.604547 7044 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:13:57.604806 7044 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:13:57.607714 7044 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439250192320-29290-0/minicluster-data/master-1/data/instance:
uuid: "1a9cd664d76b4c17a990099aac7f0be8"
format_stamp: "Formatted at 2026-08-11 09:13:57 on dist-test-slave-xt4k"
server_key: "e003c3cb2c732e9f7fdb9763d265cf4b"
server_key_iv: "faa990fc4f315db2da16f90954cc03c7"
server_key_version: "encryptionkey@0"
I20260811 09:13:57.608511 7044 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439250192320-29290-0/minicluster-data/master-1/wal/instance:
uuid: "1a9cd664d76b4c17a990099aac7f0be8"
format_stamp: "Formatted at 2026-08-11 09:13:57 on dist-test-slave-xt4k"
server_key: "e003c3cb2c732e9f7fdb9763d265cf4b"
server_key_iv: "faa990fc4f315db2da16f90954cc03c7"
server_key_version: "encryptionkey@0"
I20260811 09:13:57.613576 7044 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260811 09:13:57.617044 7072 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:13:57.618369 7044 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 09:13:57.618532 7044 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439250192320-29290-0/minicluster-data/master-1/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439250192320-29290-0/minicluster-data/master-1/wal
uuid: "1a9cd664d76b4c17a990099aac7f0be8"
format_stamp: "Formatted at 2026-08-11 09:13:57 on dist-test-slave-xt4k"
server_key: "e003c3cb2c732e9f7fdb9763d265cf4b"
server_key_iv: "faa990fc4f315db2da16f90954cc03c7"
server_key_version: "encryptionkey@0"
I20260811 09:13:57.618743 7044 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439250192320-29290-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439250192320-29290-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439250192320-29290-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:13:57.645511 7044 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:13:57.649822 7044 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:13:57.650081 7044 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:13:57.667701 7044 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.189:45629
I20260811 09:13:57.667717 7175 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.189:45629 every 8 connection(s)
I20260811 09:13:57.669112 7044 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439250192320-29290-0/minicluster-data/master-1/data/info.pb
I20260811 09:13:57.672932 7176 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:13:57.678649 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 7044
I20260811 09:13:57.678865 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439250192320-29290-0/minicluster-data/master-1/wal/instance
I20260811 09:13:57.679404 29290 external_mini_cluster.cc:1428] Setting key ca29e9e1065904b555f1bd49f84fe561
I20260811 09:13:57.679931 7176 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.154.190" port: 44423 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 09:13:57 dist-test-slave-xt4k krb5kdc[6876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439637, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.189@KRBTEST.COM for kudu/127.28.154.190@KRBTEST.COM
I20260811 09:13:57.701319 7178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:57.683087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57638 (local address 127.28.154.190:44423)
0811 09:13:57.683701 (+ 614us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:57.683707 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:57.684667 (+ 960us) server_negotiation.cc:407] Connection header received
0811 09:13:57.686492 (+ 1825us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:57.686498 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:57.686580 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:57.686705 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:57.688148 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:57.689104 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:57.690634 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:57.691175 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:57.691298 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:57.694905 (+ 3607us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:13:57.694910 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:13:57.694913 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:13:57.694968 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:13:57.698132 (+ 3164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:13:57.698918 (+ 786us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:13:57.698923 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:13:57.698926 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:13:57.699080 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:13:57.699512 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:13:57.699516 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:13:57.699519 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:13:57.699858 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:13:57.699992 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:57.700758 (+ 766us) server_negotiation.cc:299] Negotiation successful
0811 09:13:57.700970 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":490,"thread_start_us":390,"threads_started":1}
I20260811 09:13:57.702210 7179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:57.683704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:44423 (local address 127.0.0.1:57638)
0811 09:13:57.684498 (+ 794us) negotiation.cc:107] Waiting for socket to connect
0811 09:13:57.684551 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 09:13:57.686234 (+ 1683us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:13:57.686878 (+ 644us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:13:57.686886 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:13:57.687388 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:13:57.687934 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:13:57.687952 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:13:57.689282 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:13:57.689287 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:13:57.690468 (+ 1181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:13:57.690481 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:13:57.691361 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:13:57.691366 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:13:57.691592 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:57.692224 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:13:57.692247 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:13:57.694639 (+ 2392us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:13:57.698300 (+ 3661us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:13:57.698309 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:13:57.698323 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:13:57.698790 (+ 467us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:13:57.699198 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:13:57.699202 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:13:57.699205 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:13:57.699403 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:13:57.699972 (+ 569us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:13:57.699978 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:13:57.700377 (+ 399us) client_negotiation.cc:769] Sending connection context
0811 09:13:57.700649 (+ 272us) client_negotiation.cc:240] Negotiation successful
0811 09:13:57.701059 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":299,"threads_started":1}
I20260811 09:13:57.704104 7176 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.154.189" port: 45629 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.28.154.188@KRBTEST.COM; defaulting to no policy
Aug 11 09:13:57 dist-test-slave-xt4k krb5kdc[6876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439637, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.189@KRBTEST.COM for kudu/127.28.154.189@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.154.188@KRBTEST.COM" created.
I20260811 09:13:57.719071 7179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:57.704792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.189:45629 (local address 127.0.0.1:41890)
0811 09:13:57.704969 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0811 09:13:57.704988 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 09:13:57.705087 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:13:57.706234 (+ 1147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:13:57.706239 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:13:57.706698 (+ 459us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:13:57.706828 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:13:57.706836 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:13:57.708191 (+ 1355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:13:57.708197 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:13:57.709135 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:13:57.709147 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:13:57.709967 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:13:57.709974 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:13:57.710165 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:57.710791 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:13:57.710815 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:13:57.713232 (+ 2417us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:13:57.716476 (+ 3244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:13:57.716482 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:13:57.716485 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:13:57.716962 (+ 477us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:13:57.717320 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:13:57.717324 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:13:57.717327 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:13:57.717499 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:13:57.718110 (+ 611us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:13:57.718115 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:13:57.718262 (+ 147us) client_negotiation.cc:769] Sending connection context
0811 09:13:57.718407 (+ 145us) client_negotiation.cc:240] Negotiation successful
0811 09:13:57.718699 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63}
I20260811 09:13:57.719120 7183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:57.705139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41890 (local address 127.28.154.189:45629)
0811 09:13:57.705798 (+ 659us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:57.705804 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:57.705830 (+ 26us) server_negotiation.cc:407] Connection header received
0811 09:13:57.705873 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:57.705900 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:57.706079 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:57.706214 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:57.706986 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:57.708059 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:57.709268 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:57.709810 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:57.709937 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:57.713444 (+ 3507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:13:57.713458 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:13:57.713502 (+ 44us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:13:57.713537 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:13:57.716303 (+ 2766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:13:57.717092 (+ 789us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:13:57.717100 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:13:57.717102 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:13:57.717223 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:13:57.717618 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:13:57.717621 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:13:57.717623 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:13:57.717987 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:13:57.718157 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:57.718569 (+ 412us) server_negotiation.cc:299] Negotiation successful
0811 09:13:57.718810 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":23,"server-negotiator.queue_time_us":513,"thread_start_us":372,"threads_started":1}
I20260811 09:13:57.721751 7176 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.154.188" port: 37749 } has no permanent_uuid. Determining permanent_uuid...
I20260811 09:13:57.722935 7179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:57.722350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.188:37749 (local address 127.0.0.1:56452)
0811 09:13:57.722587 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0811 09:13:57.722710 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.188:37749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
W20260811 09:13:57.723161 7075 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.28.154.188:37749: connect: Connection refused (error 111)
I20260811 09:13:57.724779 7179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:57.724425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.188:37749 (local address 127.0.0.1:56454)
0811 09:13:57.724571 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0811 09:13:57.724660 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.188:37749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260811 09:13:57.725013 7176 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.154.188:37749: Network error: Client connection negotiation failed: client connection to 127.28.154.188:37749: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.154.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.188.keytab.
Entry for principal kudu/127.28.154.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.188.keytab.
I20260811 09:13:57.756827 29290 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
/tmp/dist-test-tasklh2cYx/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439250192320-29290-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439250192320-29290-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439250192320-29290-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439250192320-29290-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.154.188:37749
--webserver_interface=127.28.154.188
--webserver_port=0
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.188.keytab
--principal=kudu/127.28.154.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.154.148:39893
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.154.190:44423,127.28.154.189:45629,127.28.154.188:37749
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439250192320-29290-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439250192320-29290-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439250192320-29290-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 09:13:57.774353 7176 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.154.188" port: 37749 } attempt: 1
I20260811 09:13:57.775480 7179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:57.775016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.188:37749 (local address 127.0.0.1:56462)
0811 09:13:57.775230 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0811 09:13:57.775331 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.188:37749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260811 09:13:57.776456 7179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:57.776077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.188:37749 (local address 127.0.0.1:56470)
0811 09:13:57.776233 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0811 09:13:57.776306 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.188:37749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260811 09:13:57.776650 7176 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.154.188:37749: Network error: Client connection negotiation failed: client connection to 127.28.154.188:37749: connect: Connection refused (error 111)
I20260811 09:13:57.844856 7176 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.154.188" port: 37749 } attempt: 2
I20260811 09:13:57.845805 7031 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.154.189" port: 45629 } attempt: 5
I20260811 09:13:57.846164 7179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:57.845618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.188:37749 (local address 127.0.0.1:56474)
0811 09:13:57.845856 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0811 09:13:57.845964 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.188:37749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260811 09:13:57.847301 7179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:57.846875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.188:37749 (local address 127.0.0.1:56484)
0811 09:13:57.847056 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0811 09:13:57.847146 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.188:37749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260811 09:13:57.848088 7176 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.154.188:37749: Network error: Client connection negotiation failed: client connection to 127.28.154.188:37749: connect: Connection refused (error 111)
Aug 11 09:13:57 dist-test-slave-xt4k krb5kdc[6876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439637, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.190@KRBTEST.COM for kudu/127.28.154.189@KRBTEST.COM
I20260811 09:13:57.859793 7183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:57.846648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41900 (local address 127.28.154.189:45629)
0811 09:13:57.846835 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:57.846841 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:57.846880 (+ 39us) server_negotiation.cc:407] Connection header received
0811 09:13:57.846943 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:57.846947 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:57.847026 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:57.847119 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:57.847984 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:57.849148 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:57.850285 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:57.850970 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:57.851091 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:57.854325 (+ 3234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:13:57.854330 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:13:57.854333 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:13:57.854365 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:13:57.857095 (+ 2730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:13:57.857925 (+ 830us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:13:57.857930 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:13:57.857932 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:13:57.858072 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:13:57.858448 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:13:57.858451 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:13:57.858453 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:13:57.858849 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:13:57.858979 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:57.859404 (+ 425us) server_negotiation.cc:299] Negotiation successful
0811 09:13:57.859575 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 09:13:57.860031 7033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:57.846399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.189:45629 (local address 127.0.0.1:41900)
0811 09:13:57.846657 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0811 09:13:57.846675 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 09:13:57.846798 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:13:57.847184 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:13:57.847189 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:13:57.847612 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:13:57.847804 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:13:57.847814 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:13:57.849285 (+ 1471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:13:57.849289 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:13:57.850136 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:13:57.850148 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:13:57.851131 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:13:57.851136 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:13:57.851304 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:57.851883 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:13:57.851899 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:13:57.854123 (+ 2224us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:13:57.857254 (+ 3131us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:13:57.857260 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:13:57.857264 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:13:57.857806 (+ 542us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:13:57.858178 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:13:57.858183 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:13:57.858187 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:13:57.858358 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:13:57.858980 (+ 622us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:13:57.858985 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:13:57.859164 (+ 179us) client_negotiation.cc:769] Sending connection context
0811 09:13:57.859328 (+ 164us) client_negotiation.cc:240] Negotiation successful
0811 09:13:57.859658 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147}
I20260811 09:13:57.861253 7031 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.154.188" port: 37749 } has no permanent_uuid. Determining permanent_uuid...
I20260811 09:13:57.862299 7033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:57.861897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.188:37749 (local address 127.0.0.1:56490)
0811 09:13:57.862090 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0811 09:13:57.862167 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.188:37749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260811 09:13:57.863214 7033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:57.862906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.188:37749 (local address 127.0.0.1:56498)
0811 09:13:57.863049 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0811 09:13:57.863122 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.188:37749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260811 09:13:57.863399 7031 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.154.188:37749: Network error: Client connection negotiation failed: client connection to 127.28.154.188:37749: connect: Connection refused (error 111)
I20260811 09:13:57.914621 7031 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.154.188" port: 37749 } attempt: 1
I20260811 09:13:57.915819 7033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:57.915357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.188:37749 (local address 127.0.0.1:56514)
0811 09:13:57.915556 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0811 09:13:57.915647 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.188:37749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260811 09:13:57.916801 7033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:57.916490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.188:37749 (local address 127.0.0.1:56516)
0811 09:13:57.916627 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0811 09:13:57.916697 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.188:37749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260811 09:13:57.917003 7031 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.154.188:37749: Network error: Client connection negotiation failed: client connection to 127.28.154.188:37749: connect: Connection refused (error 111)
I20260811 09:13:57.939309 7176 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.154.188" port: 37749 } attempt: 3
I20260811 09:13:57.940555 7179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:57.939935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.188:37749 (local address 127.0.0.1:56532)
0811 09:13:57.940198 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0811 09:13:57.940323 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.188:37749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260811 09:13:57.941826 7179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:57.941286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.188:37749 (local address 127.0.0.1:56534)
0811 09:13:57.941547 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0811 09:13:57.941660 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.188:37749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142}
W20260811 09:13:57.942071 7176 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.154.188:37749: Network error: Client connection negotiation failed: client connection to 127.28.154.188:37749: connect: Connection refused (error 111)
I20260811 09:13:57.985204 7031 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.154.188" port: 37749 } attempt: 2
I20260811 09:13:57.986476 7033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:57.985973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.188:37749 (local address 127.0.0.1:56548)
0811 09:13:57.986217 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0811 09:13:57.986311 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.188:37749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260811 09:13:57.987543 7033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:57.987183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.188:37749 (local address 127.0.0.1:56552)
0811 09:13:57.987310 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0811 09:13:57.987431 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.188:37749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260811 09:13:57.987782 7031 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.154.188:37749: Network error: Client connection negotiation failed: client connection to 127.28.154.188:37749: connect: Connection refused (error 111)
I20260811 09:13:58.085310 7176 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.154.188" port: 37749 } attempt: 4
I20260811 09:13:58.086644 7179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:58.086058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.188:37749 (local address 127.0.0.1:56568)
0811 09:13:58.086314 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0811 09:13:58.086433 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.188:37749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260811 09:13:58.087805 7179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:58.087392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.188:37749 (local address 127.0.0.1:56580)
0811 09:13:58.087568 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0811 09:13:58.087664 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.188:37749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260811 09:13:58.088119 7176 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.154.188:37749: Network error: Client connection negotiation failed: client connection to 127.28.154.188:37749: connect: Connection refused (error 111)
I20260811 09:13:58.094014 7031 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.154.188" port: 37749 } attempt: 3
I20260811 09:13:58.095363 7033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:58.094702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.188:37749 (local address 127.0.0.1:56582)
0811 09:13:58.095026 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0811 09:13:58.095140 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.188:37749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155}
I20260811 09:13:58.096602 7033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:58.096141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.188:37749 (local address 127.0.0.1:56598)
0811 09:13:58.096312 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0811 09:13:58.096457 (+ 145us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.154.188:37749: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260811 09:13:58.096885 7031 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.154.188:37749: Network error: Client connection negotiation failed: client connection to 127.28.154.188:37749: connect: Connection refused (error 111)
W20260811 09:13:58.103030 7186 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 09:13:58.103396 7186 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 09:13:58.103492 7186 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 09:13:58.103605 7186 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 09:13:58.111074 7186 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 09:13:58.111194 7186 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 09:13:58.111289 7186 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 09:13:58.111353 7186 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 09:13:58.111446 7186 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 09:13:58.121826 7186 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.154.148:39893
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439250192320-29290-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439250192320-29290-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.28.154.190:44423,127.28.154.189:45629,127.28.154.188:37749
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439250192320-29290-0/krb5kdc/kudu_127.28.154.188.keytab
--principal=kudu/127.28.154.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.154.188:37749
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439250192320-29290-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.28.154.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439250192320-29290-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 11 Aug 2026 08:08:27 UTC on ccycloud.quasar-ynhpqn.root.comops.site
build id 4579929
ASAN enabled
I20260811 09:13:58.123227 7186 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 09:13:58.124670 7186 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 09:13:58.137701 7198 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:13:58.138129 7195 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 09:13:58.137748 7201 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 09:13:58.138823 7186 server_base.cc:1034] running on GCE node
I20260811 09:13:58.139624 7186 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 09:13:58.141157 7186 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 09:13:58.142431 7186 hybrid_clock.cc:648] HybridClock initialized: now 1786439638142328 us; error 49 us; skew 500 ppm
I20260811 09:13:58.142764 7186 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 11 09:13:58 dist-test-slave-xt4k krb5kdc[6876](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439638, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 09:13:58.146072 7186 init.cc:377] Logged in from keytab as kudu/127.28.154.188@KRBTEST.COM (short username kudu)
I20260811 09:13:58.147677 7186 webserver.cc:460] Webserver started at http://127.28.154.188:35603/ using document root <none> and password file <none>
I20260811 09:13:58.148449 7186 fs_manager.cc:359] Metadata directory not provided
I20260811 09:13:58.148581 7186 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 09:13:58.148918 7186 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 09:13:58.151922 7186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439250192320-29290-0/minicluster-data/master-2/data/instance:
uuid: "84935518464544aaa801eb10a0e0188e"
format_stamp: "Formatted at 2026-08-11 09:13:58 on dist-test-slave-xt4k"
server_key: "ce8cf99db7c061da43ee2351ddd3b53d"
server_key_iv: "f4b0787d909dddf6cea6e884f857befe"
server_key_version: "encryptionkey@0"
I20260811 09:13:58.152781 7186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439250192320-29290-0/minicluster-data/master-2/wal/instance:
uuid: "84935518464544aaa801eb10a0e0188e"
format_stamp: "Formatted at 2026-08-11 09:13:58 on dist-test-slave-xt4k"
server_key: "ce8cf99db7c061da43ee2351ddd3b53d"
server_key_iv: "f4b0787d909dddf6cea6e884f857befe"
server_key_version: "encryptionkey@0"
I20260811 09:13:58.158471 7186 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.001s
I20260811 09:13:58.162753 7212 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 09:13:58.164211 7186 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 09:13:58.164427 7186 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439250192320-29290-0/minicluster-data/master-2/data,/tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439250192320-29290-0/minicluster-data/master-2/wal
uuid: "84935518464544aaa801eb10a0e0188e"
format_stamp: "Formatted at 2026-08-11 09:13:58 on dist-test-slave-xt4k"
server_key: "ce8cf99db7c061da43ee2351ddd3b53d"
server_key_iv: "f4b0787d909dddf6cea6e884f857befe"
server_key_version: "encryptionkey@0"
I20260811 09:13:58.164654 7186 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439250192320-29290-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439250192320-29290-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439250192320-29290-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 09:13:58.203383 7186 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 09:13:58.207798 7186 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 09:13:58.208074 7186 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 09:13:58.226476 7186 rpc_server.cc:307] RPC server started. Bound to: 127.28.154.188:37749
I20260811 09:13:58.226500 7304 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.154.188:37749 every 8 connection(s)
I20260811 09:13:58.227898 7186 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439250192320-29290-0/minicluster-data/master-2/data/info.pb
I20260811 09:13:58.230652 29290 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu as pid 7186
I20260811 09:13:58.230856 29290 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasklh2cYx/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786439250192320-29290-0/minicluster-data/master-2/wal/instance
I20260811 09:13:58.231438 29290 external_mini_cluster.cc:1428] Setting key e4a6d3b79dea4bf069c4097bf7f99f17
I20260811 09:13:58.232300 7308 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 09:13:58.243021 7308 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.154.190" port: 44423 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 09:13:58 dist-test-slave-xt4k krb5kdc[6876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439636, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.154.188@KRBTEST.COM
I20260811 09:13:58.259400 7312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:58.233510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56606 (local address 127.28.154.188:37749)
0811 09:13:58.234334 (+ 824us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:58.234346 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:58.234393 (+ 47us) server_negotiation.cc:407] Connection header received
0811 09:13:58.236292 (+ 1899us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:58.236327 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:58.236961 (+ 634us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:58.237426 (+ 465us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:58.239885 (+ 2459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:58.241939 (+ 2054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:58.244036 (+ 2097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:58.245162 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:58.245348 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:58.250892 (+ 5544us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:13:58.250903 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:13:58.250918 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:13:58.250963 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:13:58.254752 (+ 3789us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:13:58.255451 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:13:58.255460 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:13:58.255470 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:13:58.255639 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:13:58.256023 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:13:58.256028 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:13:58.256031 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:13:58.256810 (+ 779us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:13:58.257133 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:58.257846 (+ 713us) server_negotiation.cc:299] Negotiation successful
0811 09:13:58.258307 (+ 461us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":608,"thread_start_us":375,"threads_started":1}
Aug 11 09:13:58 dist-test-slave-xt4k krb5kdc[6876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439638, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.188@KRBTEST.COM for kudu/127.28.154.190@KRBTEST.COM
I20260811 09:13:58.269475 7321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:58.247485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57654 (local address 127.28.154.190:44423)
0811 09:13:58.247965 (+ 480us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:58.247971 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:58.248045 (+ 74us) server_negotiation.cc:407] Connection header received
0811 09:13:58.248378 (+ 333us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:58.248383 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:58.248492 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:58.248603 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:58.250188 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:58.251343 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:58.253518 (+ 2175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:58.254101 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:58.254231 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:58.263488 (+ 9257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:13:58.263496 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:13:58.263501 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:13:58.263563 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:13:58.266658 (+ 3095us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:13:58.267356 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:13:58.267360 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:13:58.267362 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:13:58.267487 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:13:58.267836 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:13:58.267841 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:13:58.267844 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:13:58.268121 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:13:58.268307 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:58.268863 (+ 556us) server_negotiation.cc:299] Negotiation successful
0811 09:13:58.269077 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":965,"threads_started":1}
I20260811 09:13:58.269487 7317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:58.247000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.190:44423 (local address 127.0.0.1:57654)
0811 09:13:58.247901 (+ 901us) negotiation.cc:107] Waiting for socket to connect
0811 09:13:58.247962 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 09:13:58.248220 (+ 258us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:13:58.248689 (+ 469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:13:58.248696 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:13:58.249614 (+ 918us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:13:58.249997 (+ 383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:13:58.250011 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:13:58.251522 (+ 1511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:13:58.251528 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:13:58.253326 (+ 1798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:13:58.253348 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:13:58.256303 (+ 2955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:13:58.256312 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:13:58.256584 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:58.258961 (+ 2377us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:13:58.258997 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:13:58.263227 (+ 4230us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:13:58.266796 (+ 3569us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:13:58.266809 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:13:58.266817 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:13:58.267241 (+ 424us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:13:58.267597 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:13:58.267602 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:13:58.267606 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:13:58.267735 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:13:58.268244 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:13:58.268252 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:13:58.268477 (+ 225us) client_negotiation.cc:769] Sending connection context
0811 09:13:58.268659 (+ 182us) client_negotiation.cc:240] Negotiation successful
0811 09:13:58.269059 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":583,"thread_start_us":640,"threads_started":1}
I20260811 09:13:58.271080 7308 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.154.189" port: 45629 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 09:13:58 dist-test-slave-xt4k krb5kdc[6876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439636, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.154.190@KRBTEST.COM
I20260811 09:13:58.274148 7031 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.154.188" port: 37749 } attempt: 4
I20260811 09:13:58.279701 7325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:58.263554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57668 (local address 127.28.154.190:44423)
0811 09:13:58.264270 (+ 716us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:58.264276 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:58.264302 (+ 26us) server_negotiation.cc:407] Connection header received
0811 09:13:58.264392 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:58.264397 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:58.264474 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:58.264568 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:58.265860 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:58.266772 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:58.267987 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:58.268616 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:58.268725 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:58.273283 (+ 4558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:13:58.273288 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:13:58.273291 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:13:58.273323 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:13:58.276908 (+ 3585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:13:58.277550 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:13:58.277557 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:13:58.277560 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:13:58.277685 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:13:58.278068 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:13:58.278073 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:13:58.278075 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:13:58.278439 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:13:58.278718 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:58.279050 (+ 332us) server_negotiation.cc:299] Negotiation successful
0811 09:13:58.279265 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":553,"thread_start_us":271,"threads_started":1}
Aug 11 09:13:58 dist-test-slave-xt4k krb5kdc[6876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439638, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.188@KRBTEST.COM for kudu/127.28.154.189@KRBTEST.COM
Aug 11 09:13:58 dist-test-slave-xt4k krb5kdc[6876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439637, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.190@KRBTEST.COM for kudu/127.28.154.188@KRBTEST.COM
I20260811 09:13:58.289377 7317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:58.271810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.189:45629 (local address 127.0.0.1:41912)
0811 09:13:58.271976 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0811 09:13:58.271994 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 09:13:58.272091 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:13:58.272570 (+ 479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:13:58.272574 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:13:58.272945 (+ 371us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:13:58.273089 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:13:58.273097 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:13:58.274564 (+ 1467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:13:58.274569 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:13:58.276026 (+ 1457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:13:58.276037 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:13:58.277637 (+ 1600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:13:58.277642 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:13:58.277831 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:58.278460 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:13:58.278482 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:13:58.281392 (+ 2910us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:13:58.286398 (+ 5006us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:13:58.286403 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:13:58.286407 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:13:58.286996 (+ 589us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:13:58.287501 (+ 505us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:13:58.287505 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:13:58.287508 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:13:58.287686 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:13:58.288389 (+ 703us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:13:58.288394 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:13:58.288568 (+ 174us) client_negotiation.cc:769] Sending connection context
0811 09:13:58.288717 (+ 149us) client_negotiation.cc:240] Negotiation successful
0811 09:13:58.289019 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260811 09:13:58.290289 7183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:58.272050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41912 (local address 127.28.154.189:45629)
0811 09:13:58.272244 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:58.272248 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:58.272263 (+ 15us) server_negotiation.cc:407] Connection header received
0811 09:13:58.272310 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:58.272315 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:58.272437 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:58.272529 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:58.273244 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:58.274393 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:58.276155 (+ 1762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:58.276652 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:58.276767 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:58.281654 (+ 4887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:13:58.281661 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:13:58.281666 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:13:58.281719 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:13:58.286230 (+ 4511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:13:58.287168 (+ 938us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:13:58.287174 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:13:58.287178 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:13:58.287355 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:13:58.287806 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:13:58.287812 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:13:58.287815 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:13:58.288236 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:13:58.288411 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:58.289740 (+ 1329us) server_negotiation.cc:299] Negotiation successful
0811 09:13:58.289922 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":76}
I20260811 09:13:58.290553 7312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:58.275304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56610 (local address 127.28.154.188:37749)
0811 09:13:58.275486 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:58.275492 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:58.275510 (+ 18us) server_negotiation.cc:407] Connection header received
0811 09:13:58.275556 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:58.275564 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:58.275631 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:58.275721 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:58.276771 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:58.277456 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:58.279175 (+ 1719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:58.279952 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:58.280166 (+ 214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:58.283968 (+ 3802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:13:58.283972 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:13:58.283976 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:13:58.284005 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:13:58.287430 (+ 3425us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:13:58.288285 (+ 855us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:13:58.288290 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:13:58.288293 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:13:58.288512 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:13:58.288917 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:13:58.288922 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:13:58.288926 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:13:58.289283 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:13:58.289434 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:58.290031 (+ 597us) server_negotiation.cc:299] Negotiation successful
0811 09:13:58.290255 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 09:13:58.291342 7308 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.154.188" port: 37749 } has no permanent_uuid. Determining permanent_uuid...
I20260811 09:13:58.291447 7033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:58.274792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.188:37749 (local address 127.0.0.1:56610)
0811 09:13:58.275014 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0811 09:13:58.275033 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 09:13:58.275141 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:13:58.275785 (+ 644us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:13:58.275791 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:13:58.276412 (+ 621us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:13:58.276620 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:13:58.276634 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:13:58.277599 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:13:58.277605 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:13:58.278997 (+ 1392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:13:58.279021 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:13:58.280629 (+ 1608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:13:58.280635 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:13:58.280845 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:58.281360 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:13:58.281382 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:13:58.283786 (+ 2404us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:13:58.287618 (+ 3832us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:13:58.287624 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:13:58.287628 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:13:58.288121 (+ 493us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:13:58.288624 (+ 503us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:13:58.288630 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:13:58.288634 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:13:58.288799 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:13:58.289420 (+ 621us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:13:58.289425 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:13:58.289752 (+ 327us) client_negotiation.cc:769] Sending connection context
0811 09:13:58.289914 (+ 162us) client_negotiation.cc:240] Negotiation successful
0811 09:13:58.290215 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":71}
Aug 11 09:13:58 dist-test-slave-xt4k krb5kdc[6876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439636, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.154.189@KRBTEST.COM
I20260811 09:13:58.298696 7031 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3ad7aeb39cf4daeadc0a36faa88710e: Bootstrap starting.
Aug 11 09:13:58 dist-test-slave-xt4k krb5kdc[6876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439638, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.188@KRBTEST.COM for kudu/127.28.154.188@KRBTEST.COM
I20260811 09:13:58.300031 7326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:58.281623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41916 (local address 127.28.154.189:45629)
0811 09:13:58.282317 (+ 694us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:58.282322 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:58.282930 (+ 608us) server_negotiation.cc:407] Connection header received
0811 09:13:58.283067 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:58.283073 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:58.283155 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:58.283256 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:58.285170 (+ 1914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:58.286191 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:58.287399 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:58.287866 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:58.288071 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:58.293758 (+ 5687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:13:58.293766 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:13:58.293771 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:13:58.293806 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:13:58.297345 (+ 3539us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:13:58.298047 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:13:58.298054 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:13:58.298058 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:13:58.298194 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:13:58.298570 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:13:58.298575 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:13:58.298579 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:13:58.298882 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:13:58.299127 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:58.299439 (+ 312us) server_negotiation.cc:299] Negotiation successful
0811 09:13:58.299668 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":37,"server-negotiator.queue_time_us":542,"thread_start_us":283,"threads_started":1}
I20260811 09:13:58.303610 7031 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d3ad7aeb39cf4daeadc0a36faa88710e: Neither blocks nor log segments found. Creating new log.
I20260811 09:13:58.305152 7031 log.cc:824] T 00000000000000000000000000000000 P d3ad7aeb39cf4daeadc0a36faa88710e: Log is configured to *not* fsync() on all Append() calls
I20260811 09:13:58.305269 29290 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 09:13:58.306506 7317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:58.292048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.188:37749 (local address 127.0.0.1:56626)
0811 09:13:58.292201 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0811 09:13:58.292219 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 09:13:58.292311 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:13:58.292779 (+ 468us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:13:58.292784 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:13:58.293243 (+ 459us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:13:58.293415 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:13:58.293426 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:13:58.294725 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:13:58.294730 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:13:58.296004 (+ 1274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:13:58.296020 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:13:58.297045 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:13:58.297051 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:13:58.297251 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:58.297775 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:13:58.297797 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:13:58.300305 (+ 2508us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:13:58.303669 (+ 3364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:13:58.303675 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:13:58.303679 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:13:58.304101 (+ 422us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:13:58.304480 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:13:58.304485 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:13:58.304488 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:13:58.304614 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:13:58.305512 (+ 898us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:13:58.305517 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:13:58.305684 (+ 167us) client_negotiation.cc:769] Sending connection context
0811 09:13:58.305829 (+ 145us) client_negotiation.cc:240] Negotiation successful
0811 09:13:58.306145 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260811 09:13:58.307322 7312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:58.292249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56626 (local address 127.28.154.188:37749)
0811 09:13:58.292438 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:58.292443 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:58.292459 (+ 16us) server_negotiation.cc:407] Connection header received
0811 09:13:58.292503 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:58.292508 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:58.292565 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:58.292640 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:58.293567 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:58.294588 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:58.296188 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:58.296851 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:58.297002 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:58.300529 (+ 3527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:13:58.300535 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:13:58.300539 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:13:58.300573 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:13:58.303550 (+ 2977us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:13:58.304203 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:13:58.304209 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:13:58.304213 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:13:58.304367 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:13:58.304726 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:13:58.304731 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:13:58.304734 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:13:58.305368 (+ 634us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:13:58.305531 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:58.306793 (+ 1262us) server_negotiation.cc:299] Negotiation successful
0811 09:13:58.307008 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 09:13:58.308954 7031 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3ad7aeb39cf4daeadc0a36faa88710e: No bootstrap required, opened a new log
I20260811 09:13:58.308241 6981 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "d3ad7aeb39cf4daeadc0a36faa88710e"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57668
I20260811 09:13:58.312965 7308 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84935518464544aaa801eb10a0e0188e: Bootstrap starting.
I20260811 09:13:58.316244 7308 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 84935518464544aaa801eb10a0e0188e: Neither blocks nor log segments found. Creating new log.
I20260811 09:13:58.317571 7308 log.cc:824] T 00000000000000000000000000000000 P 84935518464544aaa801eb10a0e0188e: Log is configured to *not* fsync() on all Append() calls
I20260811 09:13:58.317751 7031 raft_consensus.cc:348] T 00000000000000000000000000000000 P d3ad7aeb39cf4daeadc0a36faa88710e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3ad7aeb39cf4daeadc0a36faa88710e" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 44423 } } peers { permanent_uuid: "1a9cd664d76b4c17a990099aac7f0be8" member_type: VOTER last_known_addr { host: "127.28.154.189" port: 45629 } } peers { permanent_uuid: "84935518464544aaa801eb10a0e0188e" member_type: VOTER last_known_addr { host: "127.28.154.188" port: 37749 } }
I20260811 09:13:58.318184 7031 raft_consensus.cc:729] T 00000000000000000000000000000000 P d3ad7aeb39cf4daeadc0a36faa88710e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3ad7aeb39cf4daeadc0a36faa88710e, State: Initialized, Role: FOLLOWER
I20260811 09:13:58.318717 7031 consensus_queue.cc:260] T 00000000000000000000000000000000 P d3ad7aeb39cf4daeadc0a36faa88710e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3ad7aeb39cf4daeadc0a36faa88710e" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 44423 } } peers { permanent_uuid: "1a9cd664d76b4c17a990099aac7f0be8" member_type: VOTER last_known_addr { host: "127.28.154.189" port: 45629 } } peers { permanent_uuid: "84935518464544aaa801eb10a0e0188e" member_type: VOTER last_known_addr { host: "127.28.154.188" port: 37749 } }
I20260811 09:13:58.320034 7329 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3ad7aeb39cf4daeadc0a36faa88710e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3ad7aeb39cf4daeadc0a36faa88710e" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 44423 } } peers { permanent_uuid: "1a9cd664d76b4c17a990099aac7f0be8" member_type: VOTER last_known_addr { host: "127.28.154.189" port: 45629 } } peers { permanent_uuid: "84935518464544aaa801eb10a0e0188e" member_type: VOTER last_known_addr { host: "127.28.154.188" port: 37749 } } }
I20260811 09:13:58.320585 7329 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3ad7aeb39cf4daeadc0a36faa88710e [sys.catalog]: This master's current role is: FOLLOWER
I20260811 09:13:58.321211 7031 sys_catalog.cc:564] T 00000000000000000000000000000000 P d3ad7aeb39cf4daeadc0a36faa88710e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:13:58.321377 7308 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 84935518464544aaa801eb10a0e0188e: No bootstrap required, opened a new log
I20260811 09:13:58.321343 6981 raft_consensus.cc:482] T 00000000000000000000000000000000 P d3ad7aeb39cf4daeadc0a36faa88710e [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260811 09:13:58.321537 6981 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d3ad7aeb39cf4daeadc0a36faa88710e [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:13:58.325263 6981 raft_consensus.cc:504] T 00000000000000000000000000000000 P d3ad7aeb39cf4daeadc0a36faa88710e [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3ad7aeb39cf4daeadc0a36faa88710e" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 44423 } } peers { permanent_uuid: "1a9cd664d76b4c17a990099aac7f0be8" member_type: VOTER last_known_addr { host: "127.28.154.189" port: 45629 } } peers { permanent_uuid: "84935518464544aaa801eb10a0e0188e" member_type: VOTER last_known_addr { host: "127.28.154.188" port: 37749 } }
I20260811 09:13:58.328326 6981 leader_election.cc:288] T 00000000000000000000000000000000 P d3ad7aeb39cf4daeadc0a36faa88710e [CANDIDATE]: Term 1 election: Requested vote from peers 1a9cd664d76b4c17a990099aac7f0be8 (127.28.154.189:45629), 84935518464544aaa801eb10a0e0188e (127.28.154.188:37749)
I20260811 09:13:58.330855 7308 raft_consensus.cc:348] T 00000000000000000000000000000000 P 84935518464544aaa801eb10a0e0188e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3ad7aeb39cf4daeadc0a36faa88710e" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 44423 } } peers { permanent_uuid: "1a9cd664d76b4c17a990099aac7f0be8" member_type: VOTER last_known_addr { host: "127.28.154.189" port: 45629 } } peers { permanent_uuid: "84935518464544aaa801eb10a0e0188e" member_type: VOTER last_known_addr { host: "127.28.154.188" port: 37749 } }
I20260811 09:13:58.331364 7308 raft_consensus.cc:729] T 00000000000000000000000000000000 P 84935518464544aaa801eb10a0e0188e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 84935518464544aaa801eb10a0e0188e, State: Initialized, Role: FOLLOWER
I20260811 09:13:58.331943 7308 consensus_queue.cc:260] T 00000000000000000000000000000000 P 84935518464544aaa801eb10a0e0188e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3ad7aeb39cf4daeadc0a36faa88710e" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 44423 } } peers { permanent_uuid: "1a9cd664d76b4c17a990099aac7f0be8" member_type: VOTER last_known_addr { host: "127.28.154.189" port: 45629 } } peers { permanent_uuid: "84935518464544aaa801eb10a0e0188e" member_type: VOTER last_known_addr { host: "127.28.154.188" port: 37749 } }
I20260811 09:13:58.333595 7344 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84935518464544aaa801eb10a0e0188e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3ad7aeb39cf4daeadc0a36faa88710e" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 44423 } } peers { permanent_uuid: "1a9cd664d76b4c17a990099aac7f0be8" member_type: VOTER last_known_addr { host: "127.28.154.189" port: 45629 } } peers { permanent_uuid: "84935518464544aaa801eb10a0e0188e" member_type: VOTER last_known_addr { host: "127.28.154.188" port: 37749 } } }
I20260811 09:13:58.334661 7308 sys_catalog.cc:564] T 00000000000000000000000000000000 P 84935518464544aaa801eb10a0e0188e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:13:58.334728 7267 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d3ad7aeb39cf4daeadc0a36faa88710e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "84935518464544aaa801eb10a0e0188e"
I20260811 09:13:58.335373 7267 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 84935518464544aaa801eb10a0e0188e [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:13:58.339488 7267 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 84935518464544aaa801eb10a0e0188e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3ad7aeb39cf4daeadc0a36faa88710e in term 1.
I20260811 09:13:58.340619 6936 leader_election.cc:302] T 00000000000000000000000000000000 P d3ad7aeb39cf4daeadc0a36faa88710e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 84935518464544aaa801eb10a0e0188e, d3ad7aeb39cf4daeadc0a36faa88710e; no voters:
I20260811 09:13:58.341233 7329 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d3ad7aeb39cf4daeadc0a36faa88710e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 09:13:58.334595 7344 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84935518464544aaa801eb10a0e0188e [sys.catalog]: This master's current role is: FOLLOWER
I20260811 09:13:58.352464 7329 raft_consensus.cc:686] T 00000000000000000000000000000000 P d3ad7aeb39cf4daeadc0a36faa88710e [term 1 LEADER]: Becoming Leader. State: Replica: d3ad7aeb39cf4daeadc0a36faa88710e, State: Running, Role: LEADER
I20260811 09:13:58.353866 7329 consensus_queue.cc:237] T 00000000000000000000000000000000 P d3ad7aeb39cf4daeadc0a36faa88710e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3ad7aeb39cf4daeadc0a36faa88710e" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 44423 } } peers { permanent_uuid: "1a9cd664d76b4c17a990099aac7f0be8" member_type: VOTER last_known_addr { host: "127.28.154.189" port: 45629 } } peers { permanent_uuid: "84935518464544aaa801eb10a0e0188e" member_type: VOTER last_known_addr { host: "127.28.154.188" port: 37749 } }
W20260811 09:13:58.361182 6939 tablet.cc:2369] T 00000000000000000000000000000000 P d3ad7aeb39cf4daeadc0a36faa88710e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 09:13:58.365016 7350 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3ad7aeb39cf4daeadc0a36faa88710e [sys.catalog]: SysCatalogTable state changed. Reason: New leader d3ad7aeb39cf4daeadc0a36faa88710e. Latest consensus state: current_term: 1 leader_uuid: "d3ad7aeb39cf4daeadc0a36faa88710e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3ad7aeb39cf4daeadc0a36faa88710e" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 44423 } } peers { permanent_uuid: "1a9cd664d76b4c17a990099aac7f0be8" member_type: VOTER last_known_addr { host: "127.28.154.189" port: 45629 } } peers { permanent_uuid: "84935518464544aaa801eb10a0e0188e" member_type: VOTER last_known_addr { host: "127.28.154.188" port: 37749 } } }
I20260811 09:13:58.366039 7350 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3ad7aeb39cf4daeadc0a36faa88710e [sys.catalog]: This master's current role is: LEADER
I20260811 09:13:58.368407 7363 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 09:13:58.371898 7366 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 09:13:58.383034 7366 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 09:13:58.373306 6902 master_runner.cc:186] Error getting master registration for 127.28.154.189:45629: OK, instance_id { permanent_uuid: "1a9cd664d76b4c17a990099aac7f0be8" instance_seqno: 1786439637662749 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 09:13:58.385061 6902 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 09:13:58.387336 7176 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.154.188" port: 37749 } attempt: 5
Aug 11 09:13:58 dist-test-slave-xt4k krb5kdc[6876](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786439637, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.154.189@KRBTEST.COM for kudu/127.28.154.188@KRBTEST.COM
I20260811 09:13:58.399101 7267 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 84935518464544aaa801eb10a0e0188e [term 1 FOLLOWER]: Refusing update from remote peer d3ad7aeb39cf4daeadc0a36faa88710e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:13:58.400400 7329 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d3ad7aeb39cf4daeadc0a36faa88710e [LEADER]: Connected to new peer: Peer: permanent_uuid: "84935518464544aaa801eb10a0e0188e" member_type: VOTER last_known_addr { host: "127.28.154.188" port: 37749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:13:58.406900 7312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:58.388086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56632 (local address 127.28.154.188:37749)
0811 09:13:58.388252 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:58.388258 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:58.388279 (+ 21us) server_negotiation.cc:407] Connection header received
0811 09:13:58.388545 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:58.388552 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:58.388637 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:58.388743 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:58.389908 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:58.391049 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:58.392998 (+ 1949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:58.393682 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:58.393807 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:58.398628 (+ 4821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:13:58.398636 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:13:58.398642 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:13:58.398684 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:13:58.402307 (+ 3623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:13:58.403503 (+ 1196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:13:58.403510 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:13:58.403514 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:13:58.403684 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:13:58.404330 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:13:58.404356 (+ 26us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:13:58.404360 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:13:58.404751 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:13:58.404895 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:58.405772 (+ 877us) server_negotiation.cc:299] Negotiation successful
0811 09:13:58.406200 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 09:13:58.408437 7179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:58.388010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.154.188:37749 (local address 127.0.0.1:56632)
0811 09:13:58.388190 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0811 09:13:58.388212 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 09:13:58.388370 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 09:13:58.388956 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 09:13:58.388961 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 09:13:58.389501 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0811 09:13:58.389713 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:13:58.389727 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:13:58.391221 (+ 1494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:13:58.391227 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:13:58.392788 (+ 1561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 09:13:58.392806 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 09:13:58.393846 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 09:13:58.393853 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 09:13:58.394107 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:58.394826 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 09:13:58.394870 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0811 09:13:58.398360 (+ 3490us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 09:13:58.402533 (+ 4173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:13:58.402539 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:13:58.402544 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:13:58.403335 (+ 791us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:13:58.403863 (+ 528us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 09:13:58.403868 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 09:13:58.403872 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 09:13:58.404170 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 09:13:58.405017 (+ 847us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 09:13:58.405022 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 09:13:58.405268 (+ 246us) client_negotiation.cc:769] Sending connection context
0811 09:13:58.407274 (+ 2006us) client_negotiation.cc:240] Negotiation successful
0811 09:13:58.407800 (+ 526us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260811 09:13:58.414798 7176 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a9cd664d76b4c17a990099aac7f0be8: Bootstrap starting.
I20260811 09:13:58.421556 7176 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1a9cd664d76b4c17a990099aac7f0be8: Neither blocks nor log segments found. Creating new log.
I20260811 09:13:58.423336 7344 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84935518464544aaa801eb10a0e0188e [sys.catalog]: SysCatalogTable state changed. Reason: New leader d3ad7aeb39cf4daeadc0a36faa88710e. Latest consensus state: current_term: 1 leader_uuid: "d3ad7aeb39cf4daeadc0a36faa88710e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3ad7aeb39cf4daeadc0a36faa88710e" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 44423 } } peers { permanent_uuid: "1a9cd664d76b4c17a990099aac7f0be8" member_type: VOTER last_known_addr { host: "127.28.154.189" port: 45629 } } peers { permanent_uuid: "84935518464544aaa801eb10a0e0188e" member_type: VOTER last_known_addr { host: "127.28.154.188" port: 37749 } } }
I20260811 09:13:58.423666 7344 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84935518464544aaa801eb10a0e0188e [sys.catalog]: This master's current role is: FOLLOWER
I20260811 09:13:58.424645 7176 log.cc:824] T 00000000000000000000000000000000 P 1a9cd664d76b4c17a990099aac7f0be8: Log is configured to *not* fsync() on all Append() calls
I20260811 09:13:58.426191 7325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:58.382526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57676 (local address 127.28.154.190:44423)
0811 09:13:58.382790 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:58.382797 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:58.400890 (+ 18093us) server_negotiation.cc:407] Connection header received
0811 09:13:58.401164 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:58.401169 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:58.401257 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:58.401325 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:58.410538 (+ 9213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:58.411659 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:58.413526 (+ 1867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:58.414185 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:58.414298 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:58.418202 (+ 3904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:13:58.418209 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:13:58.418214 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:13:58.418257 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:13:58.422034 (+ 3777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:13:58.423035 (+ 1001us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:13:58.423041 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:13:58.423046 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:13:58.423209 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:13:58.423897 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:13:58.423903 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:13:58.423906 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:13:58.424531 (+ 625us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:13:58.424807 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:58.425513 (+ 706us) server_negotiation.cc:299] Negotiation successful
0811 09:13:58.425784 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 09:13:58.431536 7393 mvcc.cc:205] Tried to move back new op lower bound from 7317256758868733952 to 7317256758711574528. Current Snapshot: MvccSnapshot[applied={T|T < 7317256758868733952}]
I20260811 09:13:58.431734 7379 mvcc.cc:205] Tried to move back new op lower bound from 7317256758868733952 to 7317256758711574528. Current Snapshot: MvccSnapshot[applied={T|T < 7317256758868733952}]
I20260811 09:13:58.434782 7329 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3ad7aeb39cf4daeadc0a36faa88710e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d3ad7aeb39cf4daeadc0a36faa88710e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3ad7aeb39cf4daeadc0a36faa88710e" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 44423 } } peers { permanent_uuid: "1a9cd664d76b4c17a990099aac7f0be8" member_type: VOTER last_known_addr { host: "127.28.154.189" port: 45629 } } peers { permanent_uuid: "84935518464544aaa801eb10a0e0188e" member_type: VOTER last_known_addr { host: "127.28.154.188" port: 37749 } } }
I20260811 09:13:58.435092 7329 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3ad7aeb39cf4daeadc0a36faa88710e [sys.catalog]: This master's current role is: LEADER
I20260811 09:13:58.443329 7366 catalog_manager.cc:1347] Generated new cluster ID: 9bec71237af64b1a94271c12ffe0cf92
I20260811 09:13:58.443419 7366 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 09:13:58.443267 7344 sys_catalog.cc:455] T 00000000000000000000000000000000 P 84935518464544aaa801eb10a0e0188e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d3ad7aeb39cf4daeadc0a36faa88710e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3ad7aeb39cf4daeadc0a36faa88710e" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 44423 } } peers { permanent_uuid: "1a9cd664d76b4c17a990099aac7f0be8" member_type: VOTER last_known_addr { host: "127.28.154.189" port: 45629 } } peers { permanent_uuid: "84935518464544aaa801eb10a0e0188e" member_type: VOTER last_known_addr { host: "127.28.154.188" port: 37749 } } }
I20260811 09:13:58.443552 7344 sys_catalog.cc:458] T 00000000000000000000000000000000 P 84935518464544aaa801eb10a0e0188e [sys.catalog]: This master's current role is: FOLLOWER
I20260811 09:13:58.434557 7186 master_runner.cc:186] Error getting master registration for 127.28.154.189:45629: OK, instance_id { permanent_uuid: "1a9cd664d76b4c17a990099aac7f0be8" instance_seqno: 1786439637662749 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 09:13:58.452728 7186 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 09:13:58.456580 7176 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a9cd664d76b4c17a990099aac7f0be8: No bootstrap required, opened a new log
I20260811 09:13:58.463718 7326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:58.401011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41920 (local address 127.28.154.189:45629)
0811 09:13:58.401196 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:58.401202 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:58.416029 (+ 14827us) server_negotiation.cc:407] Connection header received
0811 09:13:58.416312 (+ 283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:58.416318 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:58.416440 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:58.416539 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:58.426140 (+ 9601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:58.427160 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:58.429904 (+ 2744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:58.430528 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:58.430639 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:58.439061 (+ 8422us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:13:58.439070 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:13:58.439076 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:13:58.439127 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:13:58.443016 (+ 3889us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:13:58.460836 (+ 17820us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:13:58.460843 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:13:58.460847 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:13:58.461059 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:13:58.461658 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:13:58.461668 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:13:58.461671 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:13:58.462113 (+ 442us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:13:58.462436 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:58.463065 (+ 629us) server_negotiation.cc:299] Negotiation successful
0811 09:13:58.463375 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 09:13:58.464234 7366 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 09:13:58.466681 7397 catalog_manager.cc:1259] Loaded cluster ID: 9bec71237af64b1a94271c12ffe0cf92
I20260811 09:13:58.466768 7397 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 84935518464544aaa801eb10a0e0188e: loading cluster ID for follower catalog manager: success
I20260811 09:13:58.469941 7176 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1a9cd664d76b4c17a990099aac7f0be8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3ad7aeb39cf4daeadc0a36faa88710e" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 44423 } } peers { permanent_uuid: "1a9cd664d76b4c17a990099aac7f0be8" member_type: VOTER last_known_addr { host: "127.28.154.189" port: 45629 } } peers { permanent_uuid: "84935518464544aaa801eb10a0e0188e" member_type: VOTER last_known_addr { host: "127.28.154.188" port: 37749 } }
I20260811 09:13:58.470407 7176 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1a9cd664d76b4c17a990099aac7f0be8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a9cd664d76b4c17a990099aac7f0be8, State: Initialized, Role: FOLLOWER
I20260811 09:13:58.470580 7397 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 84935518464544aaa801eb10a0e0188e: acquiring CA information for follower catalog manager: success
I20260811 09:13:58.470997 7176 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1a9cd664d76b4c17a990099aac7f0be8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3ad7aeb39cf4daeadc0a36faa88710e" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 44423 } } peers { permanent_uuid: "1a9cd664d76b4c17a990099aac7f0be8" member_type: VOTER last_known_addr { host: "127.28.154.189" port: 45629 } } peers { permanent_uuid: "84935518464544aaa801eb10a0e0188e" member_type: VOTER last_known_addr { host: "127.28.154.188" port: 37749 } }
I20260811 09:13:58.471490 7366 catalog_manager.cc:1504] Loading token signing keys...
W20260811 09:13:58.471979 7397 catalog_manager.cc:873] Not found: no TSK found in the system table: failed to prepare follower catalog manager, will retry
I20260811 09:13:58.478070 7402 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a9cd664d76b4c17a990099aac7f0be8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3ad7aeb39cf4daeadc0a36faa88710e" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 44423 } } peers { permanent_uuid: "1a9cd664d76b4c17a990099aac7f0be8" member_type: VOTER last_known_addr { host: "127.28.154.189" port: 45629 } } peers { permanent_uuid: "84935518464544aaa801eb10a0e0188e" member_type: VOTER last_known_addr { host: "127.28.154.188" port: 37749 } } }
I20260811 09:13:58.478572 7402 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a9cd664d76b4c17a990099aac7f0be8 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 09:13:58.479369 7312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:58.416185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56642 (local address 127.28.154.188:37749)
0811 09:13:58.416807 (+ 622us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:58.416813 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:58.427355 (+ 10542us) server_negotiation.cc:407] Connection header received
0811 09:13:58.427642 (+ 287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:58.427648 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:58.427742 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:58.427809 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:58.437349 (+ 9540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:58.438588 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:58.464523 (+ 25935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:58.465234 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:58.465358 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:58.471286 (+ 5928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 09:13:58.471299 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 09:13:58.471304 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 09:13:58.471346 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 09:13:58.475324 (+ 3978us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:13:58.476331 (+ 1007us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:13:58.476363 (+ 32us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:13:58.476367 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:13:58.476552 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 09:13:58.477045 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 09:13:58.477050 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 09:13:58.477053 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 09:13:58.477488 (+ 435us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 09:13:58.477740 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:58.478326 (+ 586us) server_negotiation.cc:299] Negotiation successful
0811 09:13:58.478864 (+ 538us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":469}
I20260811 09:13:58.480860 7136 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d3ad7aeb39cf4daeadc0a36faa88710e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "1a9cd664d76b4c17a990099aac7f0be8"
I20260811 09:13:58.481343 7131 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1a9cd664d76b4c17a990099aac7f0be8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 09:13:58.480839 7176 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1a9cd664d76b4c17a990099aac7f0be8 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 09:13:58.487627 7131 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1a9cd664d76b4c17a990099aac7f0be8 [term 1 FOLLOWER]: Refusing update from remote peer d3ad7aeb39cf4daeadc0a36faa88710e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 09:13:58.491585 7136 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1a9cd664d76b4c17a990099aac7f0be8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3ad7aeb39cf4daeadc0a36faa88710e in term 1.
I20260811 09:13:58.497902 7329 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d3ad7aeb39cf4daeadc0a36faa88710e [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a9cd664d76b4c17a990099aac7f0be8" member_type: VOTER last_known_addr { host: "127.28.154.189" port: 45629 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 09:13:58.525815 7366 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d3ad7aeb39cf4daeadc0a36faa88710e: Generated new TSK 0
I20260811 09:13:58.527014 7366 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 09:13:58.579566 7402 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a9cd664d76b4c17a990099aac7f0be8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d3ad7aeb39cf4daeadc0a36faa88710e. Latest consensus state: current_term: 1 leader_uuid: "d3ad7aeb39cf4daeadc0a36faa88710e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3ad7aeb39cf4daeadc0a36faa88710e" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 44423 } } peers { permanent_uuid: "1a9cd664d76b4c17a990099aac7f0be8" member_type: VOTER last_known_addr { host: "127.28.154.189" port: 45629 } } peers { permanent_uuid: "84935518464544aaa801eb10a0e0188e" member_type: VOTER last_known_addr { host: "127.28.154.188" port: 37749 } } }
I20260811 09:13:58.579493 7424 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a9cd664d76b4c17a990099aac7f0be8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d3ad7aeb39cf4daeadc0a36faa88710e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3ad7aeb39cf4daeadc0a36faa88710e" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 44423 } } peers { permanent_uuid: "1a9cd664d76b4c17a990099aac7f0be8" member_type: VOTER last_known_addr { host: "127.28.154.189" port: 45629 } } peers { permanent_uuid: "84935518464544aaa801eb10a0e0188e" member_type: VOTER last_known_addr { host: "127.28.154.188" port: 37749 } } }
I20260811 09:13:58.579927 7402 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a9cd664d76b4c17a990099aac7f0be8 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 09:13:58.579993 7424 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a9cd664d76b4c17a990099aac7f0be8 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 09:13:58.581394 7350 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3ad7aeb39cf4daeadc0a36faa88710e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d3ad7aeb39cf4daeadc0a36faa88710e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3ad7aeb39cf4daeadc0a36faa88710e" member_type: VOTER last_known_addr { host: "127.28.154.190" port: 44423 } } peers { permanent_uuid: "1a9cd664d76b4c17a990099aac7f0be8" member_type: VOTER last_known_addr { host: "127.28.154.189" port: 45629 } } peers { permanent_uuid: "84935518464544aaa801eb10a0e0188e" member_type: VOTER last_known_addr { host: "127.28.154.188" port: 37749 } } }
I20260811 09:13:58.581666 7350 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3ad7aeb39cf4daeadc0a36faa88710e [sys.catalog]: This master's current role is: LEADER
I20260811 09:13:58.627527 7440 catalog_manager.cc:1259] Loaded cluster ID: 9bec71237af64b1a94271c12ffe0cf92
I20260811 09:13:58.627669 7440 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1a9cd664d76b4c17a990099aac7f0be8: loading cluster ID for follower catalog manager: success
I20260811 09:13:58.631747 7440 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1a9cd664d76b4c17a990099aac7f0be8: acquiring CA information for follower catalog manager: success
I20260811 09:13:58.634442 7440 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1a9cd664d76b4c17a990099aac7f0be8: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260811 09:13:58.683835 7325 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:13:58.672323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57682 (local address 127.28.154.190:44423)
0811 09:13:58.672708 (+ 385us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:58.672715 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:58.681746 (+ 9031us) server_negotiation.cc:407] Connection header received
0811 09:13:58.682077 (+ 331us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:58.682084 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:58.682185 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:58.682261 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:58.683556 (+ 1295us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57682: BlockingRecv error: recv got EOF from 127.0.0.1:57682 (error 108)
Metrics: {"server-negotiator.queue_time_us":100}
W20260811 09:13:58.687780 7326 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:13:58.681056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41936 (local address 127.28.154.189:45629)
0811 09:13:58.681226 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:58.681233 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:58.685522 (+ 4289us) server_negotiation.cc:407] Connection header received
0811 09:13:58.686281 (+ 759us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:58.686287 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:58.686397 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:58.686471 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:58.687521 (+ 1050us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41936: BlockingRecv error: recv got EOF from 127.0.0.1:41936 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
W20260811 09:13:58.693517 7312 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:13:58.690758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56650 (local address 127.28.154.188:37749)
0811 09:13:58.690925 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:58.690930 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:58.691524 (+ 594us) server_negotiation.cc:407] Connection header received
0811 09:13:58.691776 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:58.691783 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:58.691905 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:58.692011 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:58.693246 (+ 1235us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56650: BlockingRecv error: recv got EOF from 127.0.0.1:56650 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 09:13:58.742880 7326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:58.725940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41946 (local address 127.28.154.189:45629)
0811 09:13:58.726134 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:58.726140 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:58.734216 (+ 8076us) server_negotiation.cc:407] Connection header received
0811 09:13:58.734781 (+ 565us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:58.734788 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:58.735041 (+ 253us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:58.735204 (+ 163us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:13:58.735955 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:58.737317 (+ 1362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:58.739570 (+ 2253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:58.740258 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:58.740417 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:58.741200 (+ 783us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:13:58.741984 (+ 784us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:13:58.742111 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:58.742393 (+ 282us) server_negotiation.cc:299] Negotiation successful
0811 09:13:58.742510 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
W20260811 09:13:58.787931 7312 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:13:58.777344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56652 (local address 127.28.154.188:37749)
0811 09:13:58.777575 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:58.777582 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:58.785500 (+ 7918us) server_negotiation.cc:407] Connection header received
0811 09:13:58.786161 (+ 661us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:58.786169 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:58.786355 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:58.786452 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:58.787680 (+ 1228us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56652: BlockingRecv error: recv got EOF from 127.0.0.1:56652 (error 108)
Metrics: {"server-negotiator.queue_time_us":95}
I20260811 09:13:58.847280 7326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:58.827812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41954 (local address 127.28.154.189:45629)
0811 09:13:58.828016 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:58.828023 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:58.840035 (+ 12012us) server_negotiation.cc:407] Connection header received
0811 09:13:58.840640 (+ 605us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:58.840647 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:58.840774 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:58.840844 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:13:58.841590 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:58.842710 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:58.844764 (+ 2054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:58.845487 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:58.845598 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:58.846310 (+ 712us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:13:58.846484 (+ 174us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:13:58.846557 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:58.846821 (+ 264us) server_negotiation.cc:299] Negotiation successful
0811 09:13:58.846935 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
W20260811 09:13:58.893679 7312 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:13:58.882036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56660 (local address 127.28.154.188:37749)
0811 09:13:58.882241 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:58.882247 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:58.889631 (+ 7384us) server_negotiation.cc:407] Connection header received
0811 09:13:58.889886 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:58.889891 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:58.889992 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:58.890096 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:58.893421 (+ 3325us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56660: BlockingRecv error: recv got EOF from 127.0.0.1:56660 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 09:13:58.950730 7326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:58.935243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41960 (local address 127.28.154.189:45629)
0811 09:13:58.935536 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:58.935543 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:58.943201 (+ 7658us) server_negotiation.cc:407] Connection header received
0811 09:13:58.943750 (+ 549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:58.943757 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:58.943864 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:58.943948 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:13:58.944977 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:58.946121 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:58.948249 (+ 2128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:58.949010 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:58.949125 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:58.949791 (+ 666us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:13:58.949968 (+ 177us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:13:58.950040 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:58.950305 (+ 265us) server_negotiation.cc:299] Negotiation successful
0811 09:13:58.950406 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108}
W20260811 09:13:58.990938 7312 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:13:58.981129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56668 (local address 127.28.154.188:37749)
0811 09:13:58.981353 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:58.981359 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:58.988851 (+ 7492us) server_negotiation.cc:407] Connection header received
0811 09:13:58.989378 (+ 527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:58.989385 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:58.989491 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:58.989566 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:58.990668 (+ 1102us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56668: BlockingRecv error: recv got EOF from 127.0.0.1:56668 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 09:13:59.037681 7326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:59.031008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41976 (local address 127.28.154.189:45629)
0811 09:13:59.031229 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:59.031235 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:59.032101 (+ 866us) server_negotiation.cc:407] Connection header received
0811 09:13:59.032298 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:59.032303 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:59.032419 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:59.032515 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:13:59.033218 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:59.034110 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:59.035650 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:59.036267 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:59.036409 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:59.036885 (+ 476us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:13:59.037043 (+ 158us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:13:59.037175 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:59.037306 (+ 131us) server_negotiation.cc:299] Negotiation successful
0811 09:13:59.037390 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
W20260811 09:13:59.058382 7312 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:13:59.054424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56682 (local address 127.28.154.188:37749)
0811 09:13:59.054664 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:59.054668 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:59.057006 (+ 2338us) server_negotiation.cc:407] Connection header received
0811 09:13:59.057205 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:59.057212 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:59.057295 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:59.057392 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:59.058187 (+ 795us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56682: BlockingRecv error: recv got EOF from 127.0.0.1:56682 (error 108)
Metrics: {"server-negotiator.queue_time_us":125}
I20260811 09:13:59.075628 7326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:59.070319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41984 (local address 127.28.154.189:45629)
0811 09:13:59.070551 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:59.070558 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:59.070882 (+ 324us) server_negotiation.cc:407] Connection header received
0811 09:13:59.071092 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:59.071097 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:59.071178 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:59.071261 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:13:59.071713 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:59.072571 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:59.073880 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:59.074328 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:59.074456 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:59.074866 (+ 410us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:13:59.074976 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:13:59.075053 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:59.075204 (+ 151us) server_negotiation.cc:299] Negotiation successful
0811 09:13:59.075298 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
W20260811 09:13:59.088651 7312 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:13:59.086297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56684 (local address 127.28.154.188:37749)
0811 09:13:59.086494 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:59.086501 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:59.086914 (+ 413us) server_negotiation.cc:407] Connection header received
0811 09:13:59.087085 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:59.087091 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:59.087210 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:59.087338 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:59.088267 (+ 929us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56684: BlockingRecv error: recv got EOF from 127.0.0.1:56684 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 09:13:59.113427 7326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:59.108006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41986 (local address 127.28.154.189:45629)
0811 09:13:59.108221 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:59.108227 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:59.108579 (+ 352us) server_negotiation.cc:407] Connection header received
0811 09:13:59.108807 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:59.108811 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:59.108884 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:59.108964 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:13:59.109404 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:59.110265 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:59.111609 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:59.112058 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:59.112197 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:59.112642 (+ 445us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:13:59.112771 (+ 129us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:13:59.112854 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:59.113023 (+ 169us) server_negotiation.cc:299] Negotiation successful
0811 09:13:59.113116 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
W20260811 09:13:59.122460 7312 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:13:59.120436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56694 (local address 127.28.154.188:37749)
0811 09:13:59.120651 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:59.120657 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:59.121017 (+ 360us) server_negotiation.cc:407] Connection header received
0811 09:13:59.121135 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:59.121142 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:59.121244 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:59.121372 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:59.122200 (+ 828us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56694: BlockingRecv error: recv got EOF from 127.0.0.1:56694 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 09:13:59.163672 7326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:59.157967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42002 (local address 127.28.154.189:45629)
0811 09:13:59.158188 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:59.158196 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:59.158814 (+ 618us) server_negotiation.cc:407] Connection header received
0811 09:13:59.159027 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:59.159032 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:59.159126 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:59.159217 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:13:59.159695 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:59.160572 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:59.161947 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:59.162383 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:59.162503 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:59.162914 (+ 411us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:13:59.163027 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:13:59.163103 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:59.163257 (+ 154us) server_negotiation.cc:299] Negotiation successful
0811 09:13:59.163344 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
W20260811 09:13:59.172231 7312 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:13:59.170280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56696 (local address 127.28.154.188:37749)
0811 09:13:59.170459 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:59.170466 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:59.170822 (+ 356us) server_negotiation.cc:407] Connection header received
0811 09:13:59.170992 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:59.170998 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:59.171091 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:59.171199 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:59.171950 (+ 751us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56696: BlockingRecv error: recv got EOF from 127.0.0.1:56696 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 09:13:59.245337 7326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:59.239512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42008 (local address 127.28.154.189:45629)
0811 09:13:59.239746 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:59.239752 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:59.240153 (+ 401us) server_negotiation.cc:407] Connection header received
0811 09:13:59.240317 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:59.240322 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:59.240434 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:59.240527 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:13:59.241125 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:59.242161 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:59.243570 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:59.244024 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:59.244142 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:59.244610 (+ 468us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:13:59.244738 (+ 128us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:13:59.244815 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:59.244971 (+ 156us) server_negotiation.cc:299] Negotiation successful
0811 09:13:59.245058 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
W20260811 09:13:59.252998 7312 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:13:59.251150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56712 (local address 127.28.154.188:37749)
0811 09:13:59.251363 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:59.251370 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:59.251675 (+ 305us) server_negotiation.cc:407] Connection header received
0811 09:13:59.251828 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:59.251832 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:59.251903 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:59.251990 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:59.252800 (+ 810us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56712: BlockingRecv error: recv got EOF from 127.0.0.1:56712 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 09:13:59.390054 7326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:59.384742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42016 (local address 127.28.154.189:45629)
0811 09:13:59.384932 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:59.384940 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:59.385322 (+ 382us) server_negotiation.cc:407] Connection header received
0811 09:13:59.385547 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:59.385552 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:59.385627 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:59.385708 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:13:59.386167 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:59.386985 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:59.388383 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:59.388822 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:59.388956 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:59.389363 (+ 407us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:13:59.389483 (+ 120us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:13:59.389558 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:59.389707 (+ 149us) server_negotiation.cc:299] Negotiation successful
0811 09:13:59.389794 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
W20260811 09:13:59.397503 7312 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 09:13:59.395624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56726 (local address 127.28.154.188:37749)
0811 09:13:59.395856 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:59.395862 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:59.396169 (+ 307us) server_negotiation.cc:407] Connection header received
0811 09:13:59.396376 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:59.396382 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:59.396482 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:59.396568 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 09:13:59.397311 (+ 743us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56726: BlockingRecv error: recv got EOF from 127.0.0.1:56726 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
I20260811 09:13:59.474732 7397 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 84935518464544aaa801eb10a0e0188e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 09:13:59.662186 7326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:59.657178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42024 (local address 127.28.154.189:45629)
0811 09:13:59.657397 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:59.657403 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:59.657654 (+ 251us) server_negotiation.cc:407] Connection header received
0811 09:13:59.657810 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:59.657814 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:59.657913 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:59.658000 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:13:59.658412 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:59.659170 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:59.660540 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:59.660945 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:59.661066 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:59.661493 (+ 427us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:13:59.661610 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:13:59.661701 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:59.661815 (+ 114us) server_negotiation.cc:299] Negotiation successful
0811 09:13:59.661884 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 09:13:59.674495 7312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:59.668055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56734 (local address 127.28.154.188:37749)
0811 09:13:59.668242 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:59.668248 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:59.668671 (+ 423us) server_negotiation.cc:407] Connection header received
0811 09:13:59.668851 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:59.668855 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:59.669018 (+ 163us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:59.669151 (+ 133us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:13:59.669620 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:59.670354 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:59.672213 (+ 1859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:59.672674 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:59.672773 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:59.673345 (+ 572us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:13:59.673881 (+ 536us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:13:59.673993 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:59.674109 (+ 116us) server_negotiation.cc:299] Negotiation successful
0811 09:13:59.674181 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 09:13:59.687845 7326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:59.681403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42040 (local address 127.28.154.189:45629)
0811 09:13:59.681620 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:59.681627 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:59.682172 (+ 545us) server_negotiation.cc:407] Connection header received
0811 09:13:59.682488 (+ 316us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:59.682494 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:59.682616 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:59.682680 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:13:59.683336 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:59.684058 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:59.686018 (+ 1960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:59.686446 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:59.686568 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:59.687081 (+ 513us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:13:59.687205 (+ 124us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:13:59.687256 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:59.687456 (+ 200us) server_negotiation.cc:299] Negotiation successful
0811 09:13:59.687554 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260811 09:13:59.689767 7312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:59.682263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56740 (local address 127.28.154.188:37749)
0811 09:13:59.682475 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:59.682481 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:59.683120 (+ 639us) server_negotiation.cc:407] Connection header received
0811 09:13:59.683346 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:59.683352 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:59.683447 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:59.683524 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:13:59.684263 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:59.685351 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:59.687546 (+ 2195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:59.688284 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:59.688414 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:59.688886 (+ 472us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:13:59.689039 (+ 153us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:13:59.689134 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:59.689364 (+ 230us) server_negotiation.cc:299] Negotiation successful
0811 09:13:59.689440 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 09:13:59.698035 7650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 09:13:59.680739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57696 (local address 127.28.154.190:44423)
0811 09:13:59.692719 (+ 11980us) server_negotiation.cc:206] Beginning negotiation
0811 09:13:59.692724 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 09:13:59.692742 (+ 18us) server_negotiation.cc:407] Connection header received
0811 09:13:59.692841 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 09:13:59.692845 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 09:13:59.693014 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 09:13:59.693175 (+ 161us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 09:13:59.693664 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:59.694444 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:59.695914 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 09:13:59.696404 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 09:13:59.696556 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 09:13:59.697076 (+ 520us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 09:13:59.697454 (+ 378us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 09:13:59.697542 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 09:13:59.697643 (+ 101us) server_negotiation.cc:299] Negotiation successful
0811 09:13:59.697732 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11865,"thread_start_us":11744,"threads_started":1}
I20260811 09:13:59.702575 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 6902
I20260811 09:13:59.703588 7024 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:13:59.814110 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 6902
I20260811 09:13:59.842123 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 7044
I20260811 09:13:59.843298 7169 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:13:59.950596 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 7044
I20260811 09:13:59.976567 29290 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu pid 7186
I20260811 09:13:59.977834 7300 generic_service.cc:196] Checking for leaks (request via RPC)
I20260811 09:14:00.086594 29290 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasklh2cYx/build/asan/bin/kudu with pid 7186
2026-08-11T09:14:00Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (5083 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (10064 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (389789 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST